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FOREWORD 


Despite  significant  budgetary  restraints,  the  U.S.  Army  faces  the  challenges  of 
maintaining  combat  readiness  and  preparing  for  the  battlefield  of  the  21st  century.  In  fiscal  year 
1994,  Congress  appropriated  funding  for  the  Virtual  Brigade  Training  Program  (Department  of 
Defense,  October  1993).  Shortly  thereafter,  program  development  was  assumed  by  the  Force 
XXI  Training  Program  that  is  addressing  the  training  of  the  modem  force,  and  the  program  was 
renamed  Combined  Arms  Operations  at  the  Brigade  Level,  Realistically  Achieved  Through 
Simulation  (COBRAS). 

Several  efforts  have  been  initiated  under  the  COBRAS  umbrella,  all  funded  through  the 
Defense  Advanced  Research  Projects  Agency,  and  developed  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  and  the  U.S.  Army  Armor  Center  (Memorandum 
of  Agreement  entitled  “Force  XXI  Training  Program  [i.e.,  Virtual  Brigade  Training  Program],” 
June  1994).  The  first  two  projects,  COBRAS  I  and  II,  developed  two  types  of  exercises  designed 
for  the  staffs  of  inexperienced,  conventionally-equipped  brigades:  (1)  vignettes  for  segments  of 
the  staff  and  (2)  a  larger  Brigade  Staff  Exercise  focusing  on  the  primary  staff  members  plus  the 
staff  responsible  for  combat  support  (CS)  and  combat  service  support  (CSS).  Both  exercise  types 
offer  practice  and  feedback  opportunity  in  combat  fundamentals.  Additional  training  for 
battalion-level  CS  and  CSS  personnel  in  the  context  of  the  full  operation  was  also  developed  in 
COBRAS  II  as  an  expansion  of  the  Virtual  Training  Program,  using  virtual  simulation. 

The  third  project  in  the  overall  effort  (COBRAS  III),  the  subject  of  this  report,  expanded 
on  the  COBRAS  I  and  II  training  research  and  development,  implementing  lessons  learned  and 
satisfying  training  needs  (i.e.,  training  for  staff  personnel  linking  the  brigade  and  maneuver 
battalion  staffs  and  additional  CS  and  CSS  systems)  exposed  during  those  and  related,  concurrent 
efforts. 

This  report  discusses  the  background  of  the  COBRAS  III  project  and  documents  the 
design  and  development  of  the  resulting  training  program.  Also  contained  are  lessons  learned  for 
future  development  efforts,  and  training  issues  that  will  likely  demand  attention  in  the  near 
future.  Force  XXI  policy  makers  and  training  developers  will  find  this  report  useful  in  the  course 
of  continuing  steady  progress  toward  Force  XXI  goals  in  training  and  readiness. 


ZITA  M.  SIMUTIS 
Technical  Director 
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COMBINED  ARMS  OPERATIONS  AT  BRIGADE  LEVEL,  REALISTICALLY  ACHIEVED 
THROUGH  SIMULATION  m  (COBRAS  III):  REPORT  ON  DEVELOPMENT  AND 
LESSONS  LEARNED 


EXECUTIVE  SUMMARY  _ 

Research  Requirement: 

In  1994,  the  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  in 
coordination  with  the  Force  XXI  Training  Program  and  the  U.S.  Army  Armor  Center,  launched  a 
research  and  development  effort  designed  to  help  brigade  staffs  become  proficient  in  the  combat 
fundamentals  that  will  be  required  On  the  digital  battlefield.  This  effort,  entitled  Combined  Arms 
Operations  at  Brigade  Level  Realistically  Achieved  through  Simulation  ( COBRAS),  is 
developing  and  evaluating  structured,  simulation-based  training  programs  and  strategies  to 
address  the  training  need. 

The  two  prior  projects  (COBRAS  I  and  II)  developed  two  types  of  exercises:  a  Brigade 
Staff  Exercise  (BSE)  and  a  set  of  brigade  staff  vignettes  (Graves,  Campbell,  Deter,  &  Quinkert, 
1997;  C.H.  Campbell,  Graves,  Deter,  &  Quinkert,  1998).  The  COBRAS  HI  project  was  to 
expand  on  that  work  and  develop  two  multiechelon  exercises,  one  using  constructive  simulation 
only  and  the  other  using  the  Synthetic  Theater  of  War  (STOW)  Architecture  using  both 
constructive  and  virtual  simulation.  The  multiechelon  exercise  for  constructive  simulation  is  the 
subject  of  this  report. 

The  objectives  of  this  portion  of  the  COBRAS  ID  project  were  to: 

(a)  develop  a  constructive  simulation-based  multiechelon  Brigade  and  Battalion  Staff 
Exercise  (BBSE)  that  provides  structured  training  at  a  Combat  Training  Center 
(CTC)-like  tempo, 

(b)  focus  the  training  on  performance  involving  staffs  and  staff  sections  at  both  echelons, 

(c)  formatively  evaluate  the  newly-developed  exercise,  and 

(d)  based  on  that  that  evaluation,  refine  the  training  support  package  (TSP)  and  provide 
lessons  learned  and  insights  into  the  advancement  of  the  Force  XXI  training  strategy. 

Procedure: 

Working  from  the  COBRAS  I  and  II  products,  developers  identified  the  new  training 
audience  members,  revised  the  scenario  to  provide  performance  opportunities  for  the  full 
audience,  and  revised  the  exercise  architecture  based  on  lessons  learned  in  the  initial  projects. 
This  foundation  was  then  used  to  construct  the  TSP  materials  for  the  BBSE. 

The  comprehensive  formative  evaluation  was  integral  and  continuous,  with  numerous 
internal  and  external  reviews  of  decisions  and  products.  Two  implementations  of  the  BSE  in 
COBRAS  II  and  one  of  the  BBSE  in  COBRAS  IE  supported  improvement  to  the  quality  and 
functionality  of  the  training  model  and  TSPs. 


IX 


Findings: 

The  design,  development,  and  evaluation  processes  described  above  combined  to  produce 
structured  exercises  and  TSPs  consistent  with  project  requirements.  Products  include  this  report 
and  other  documents  relating  to  the  development  methodologies  and  the  performance 
requirements  (i.e.,  brigade  and  battalion  performance  objectives)  produced  during  the  project,  in 
addition  to  the  TSP  materials  themselves. 

The  BBSE  provides  an  opportunity  for  the  brigade  commander,  his  staff,  one  or  more 
maneuver  battalion  commanders,  and  their  staffs  to  practice  their  roles  during  all  phases  of  a 
three-mission  scenario.  The  multiechelon  training  provided  for  both  horizontal  (within  the  staffs) 
and  vertical  (between  the  brigade  and  maneuver  battalion  staffs  and  within  battlefield  operating 
systems)  integration  of  staff  groups. 

Lessons  learned  during  the  project  and  insights  into  anticipated  training  needs  are 
presented  to  assist  training  developers,  including  Force  XXI  policy-makers,  as  they  continue  to 
advance  and  promote  the  Force  XXI  Training  Program.  The  lessons  focus  on  the  development 
and  implementation  of  structured  simulation-based  training,  as  well  as  on  the  potential  value  of 
such  programs. 

Utilization  of  Findings: 

This  report  presents  the  history  of  the  development  of  the  COBRAS  HI  program  and  the 
lessons  learned  during  the  project’s  duration.  As  continued  emphasis  is  placed  on  providing  low- 
resource,  cost-effective  training  for  U.S.  Army  staff  personnel,  this  report  can  lead  those  training 
development  efforts  into  the  selection  of  effective  design  and  implementation  strategies. 
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COMBINED  ARMS  OPERATIONS  AT  BRIGADE  LEVEL,  REALISTICALLY 
ACHIEVED  THROUGH  SIMULATION  IH  (COBRAS  III): 

REPORT  ON  DEVELOPMENT  AND  LESSONS  LEARNED 

Introduction 

Military  forces  of  today  and  the  21st  century  are  at  a  decisive  point  in  how  they  do 
business.  It  is  characterized  by  the  transition  from  mostly  analog  battlefield  systems  to 
operations  with  the  new  digital  systems.  However,  before  the  transition  from  conventional  to 
digitized  systems  and  organizations  can  be  achieved.  Army  leaders  have  forecasted  the  need  to 
strengthen  basic  combat  skills,  with  an  emphasis  on  command,  control,  and  communications.  In 
the  face  of  reduced  monetary  and  personnel  resources,  they  are  looking  to  simulation-based 
training  to  complement  institutional  training  and  live  field  training  exercises  in  preparing  soldiers 
for  the  demands  of  future  missions,  organizations,  and  weapon  systems. 

One  of  the  first  such  programs  was  the  Virtual  Training  Program  (VTP)1  developed  as  a 
research  and  development  (R&D)  product  by  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  in  1993-1994.  The  VTP  provides  Army  National  Guard  and  active 
component  units  with  time-compressed  structured  training  in  virtual  and  constructive 
environments  (Hoffman,  Graves,  Roger,  Flynn,  &  Sever,  1995).  Both  offensive  and  defensive 
exercises  were  developed  for  the  battalions,  battalion  staffs,  companies,  and  platoons. 

In  1995,  ARI  undertook  another  effort,  Combined  Arms  Operations  at  Brigade  Level, 
Realistically  Achieved  through  Simulation  ( COBRAS).  The  COBRAS  project  developed 
structured  simulation-based  training  exercises  for  the  staffs  of  conventional  mounted  brigades. 
The  exercises  included  a  Brigade  Staff  Exercise  (BSE)  and  vignettes  for  subsets  of  the  brigade 
staff  (Graves,  Campbell,  Deter,  &  Quinkert,  1997). 

The  follow-on  project,  COBRAS  n,  expanded  on  those  products.  The  COBRAS  II  effort 
included  an  expanded  BSE  with  a  wider  training  audience  and  additional  brigade  staff  vignettes 
(C.H.  Campbell,  Graves,  Deter,  &  Quinkert,  1998)2. 

This  project,  COBRAS  IH,  further  expanded  the  BSE  work  through  the  development  of  a 
multiechelon  exercise  referred  to  as  the  Brigade  and  Battalion  Staff  Exercise  (BBSE),  the  subject 
of  this  report.3 


1  The  program  was  originally  known  as  the  Reserve  Component  Virtual  Training  Program.  As  active  units  and 

institutional  training  managers  became  interested  in  using  the  program,  the  “Reserve  Component”  designation  was 
dropped. 

2  The  COBRAS  II  project  also  enhanced  the  Virtual  Training  Program  (VTP)  by  adding  opportunities  for  training 

combat  support  (CS)  and  combat  service  support  (CSS)  functions.  This  COBRAS  II  battalion-level  work  is 
described  in  Hoffman’s  report,  Combat  Support  and  Combat  Service  Support  Expansion  to  the  Virtual  Training 
Program  SIMNET  Battalion  Exercise:  History  and  Lessons  Learned  (1997).  The  products  of  that  effort  reside 
with  the  VTP  observer/controller  (O/C)  team  of  the  16th  Cavalry  Regiment,  Fort  Knox. 

3  A  second  COBRAS  III  effort  used  the  Synthetic  Theater  of  War  (STOW)  Architecture  linking  both  constructive 

and  virtual  simulation,  as  the  driver  for  a  multiechelon  exercise.  It  is  referred  to  as  the  COBRAS  STOW  Exercise 
and  is  reported  in  several  ARI  reports  (COBRAS  Team,  in  preparation;  C.  H.  Campbell  et  al.,  in  preparation). 


1 


Purpose  of  the  Report 

This  report  documents  the  development  process  and  lessons  learned  during  the  BBSE 
portion  of  COBRAS  IQ  project.  It  is  addressed  to  three  audiences:  the  sponsors  of  training 
development  programs,  the  developers  of  the  training  programs,  and  those  policy-makers  who 
are  responsible  for  Army  training  today  and  in  the  future.  In  addressing  those  audiences,  the 
report  has  three  objectives: 

1.  to  detail  the  history  of  the  development  effort,  from  the  initial  design  activities 
through  the  production  and  delivery  of  the  final  training  support  package  (TSP) 
materials; 

2.  to  describe  the  resulting  training  exercises  and  TSPs,  focusing  on  the  unique  aspects 
of  the  COBRAS  HI  versus  the  COBRAS  I  and  II  program  designs;  and 

3.  to  present  lessons  learned  from  the  project,  provided  to  assist  Force  XXI  policy¬ 
makers  as  they  continue  to  advance  the  program,  and  to  aid  developers  of  other 
structured  simulation-based  training  programs. 

Organization  of  the  Report 

This  report  is  organized  in  six  sections: 

•  Section  1:  The  COBRAS  III  Project.  This  section  describes  the  early  stages  of  the 
project  in  terms  of  project  initiating  circumstances,  the  technical  objectives,  and 
training  program  design. 

•  Section  2:  Brigade  and  Battalion  Staff  Exercise  Development  and  Evaluation  Plan. 
This  section  describes  the  basic  development  methodology  and  presents  the  timeline 
of  BBSE  development  events. 

•  Section  3:  Development  of  the  Brigade  and  Battalion  Staff  Exercise.  This  section 
describes  the  development  of  the  BBSE  in  preparation  for  the  trial  implementation, 
highlighting  how  and  why  the  BSE  was  modified  to  achieve  the  objectives  for  the 
BBSE. 

•  Section  4:  Trial  Implementation  of  the  Brigade  and  Battalion  Staff  Exercise.  This 
section  describes  the  BBSE  trial  implementation  and  its  findings  and  discusses  the 
actions  taken  to  improve  the  exercise  as  a  result  of  the  trial. 

•  Section  5:  Lessons  Learned  for  the  Training  Developers.  This  section  summarizes 
lessons  learned  that  may  apply  to  future  training  development  projects.  Most  of  the 
lessons  relate  to  the  construction  and  content  of  TSPs  for  COBRAS-like  programs. 

•  Section  6:  Future  Directions.  This  section  discusses  future  development  efforts, 
including  the  next  COBRAS  project. 

Appendix  A  provides  definitions  of  the  acronyms  used  in  this  report. 
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Section  1:  The  COBRAS  III  Project 

The  planned  scope  and  activities  of  the  COBRAS  HI  project  were  outlined  by  ARI,  the 
United  States  Army  Armor  Center  (USAARMC),  and  the  Force  XXI  Training  Program  during 
the  early  stages  of  the  project.  During  this  time,  the  conceptual  foundations  of  the  program  were 
solidified  as  development  objectives  and  requirements.  This  section  describes  the  early  stages  of 
the  project,  in  terms  of  the  project  initiating  circumstances,  the  project’s  technical  objectives,  and 
the  training  program  design. 


Project  Initiation 

In  early  1994,  with  the  support  of  the  Senate  Appropriations  Committee,  the  U.S.  Army 
was  directed  to  “ . . .  expand  the  existing  simulation  facilities  at  Fort  Knox,  develop  a  training 
strategy,  use  it  to  enhance  the  readiness  of  the  194th  Separate  Armor  Brigade,  and  evaluate  the 
effectiveness  of  these  simulations  and  this  new  strategy”  (Department  of  Defense,  October  1993). 
Congressionally-identified  funding  was  provided  through  the  Defense  Advanced  Research 
Projects  Agency  to  the  ARI  -  Armored  Forces  Research  Unit  at  Fort  Knox.  The  resulting 
program,  known  as  the  Force  XXI  Training  Program,  was  seen  as  an  Army-wide  program  for 
integrating  virtual,  constructive,  and  live  simulation-based  training  into  the  Force  XXI  plan  for 
enhancing  combat  readiness.  The  effort,  conducted  by  ARI,  was  entitled  “Force  XXI  Training 
Program  for  the  Conventional  Mounted  Brigade”  and  began  in  January  1995.  One  more  name 
change  occurred  in  the  spring  of  1995,  when  the  program  title  Combined  Arms  Operations  at 
Brigade  Level,  Realistically  Achieved  Through  Simulation,  and  its  acronym,  COBRAS,  came 
into  common  usage. 

The  first  two  COBRAS  projects  (COBRAS  I  and  II)  are  the  products  of  the 
congressionally-appropriated  funding  provided  to  ARI.  The  COBRAS  II  project,  the 
continuation  of  the  original  COBRAS  project,  revised  or  replaced  all  the  COBRAS  I  products. 
The  products  delivered  in  the  COBRAS  II  project  include:  a  TSP  for  a  complex  staff  exercise  for 
the  brigade,  referred  to  as  the  BSE,  a  series  of  TSPs  for  small  group  brigade  staff  vignettes,  and 
revised  and  expanded  TSPs  for  the  VTP  battalion  exercises  adding  training  opportunities  to 
incorporate  combat  support  (CS)  and  combat  service  support  (CSS)  functions.  The  BSE  and 
VTP  battalion  expansion  are  fully  described  in  Appendix  B. 

The  COBRAS  HI  BBSE  was  derived  from  lessons  learned  after  observing  the  ways  in 
which  trial  units  modified  the  COBRAS  II  products  to  meet  their  needs  in  preparing  for  a 
National  Training  Center  (NTC)  rotation.  Again,  ARI  defined  the  requirement  in  a  statement  of 
work  ([SOW]  ARI,  1997),  and  a  technical  response  (Human  Resources  Research  Organization, 
1997)  was  prepared  to  detail  the  approach  to  meeting  the  requirement. 

Project  Purpose 

The  initial  COBRAS  VYL  work  was  designed  to  improve  the  decision-making  processes  of 
the  brigade  headquarters  through  the  use  of  simulation,  while  at  the  same  time  reducing  the 
amount  of  time  and  number  of  personnel  required  to  prepare  for  and  support  the  training.  At  a 
more  abstract  level,  however,  the  program  represented  a  test-bed  for  training  concepts  and 
training  technology.  That  is,  it  provided  an  initial  architecture  for  addressing  collective  training 
needs  for  a  conventional  brigade  staff  and  the  linkages  to  higher  and  subordinate  echelons. 


3 


As  a  training  program,  the  COBRAS  I/n  BSE  provided  the  conditions  for  brigade  staff 
members  to  practice  decision-making,  communications,  synchronization,  and  integration 
processes  among  themselves  and  with  other  echelons.  As  a  result  of  the  trials  during  the 
development  process,  three  important  indicators  of  additional  training  needs  emerged: 

•  In  the  BSE,  16  of  the  brigade  staff  leaders  were  included  as  the  primary  training 
audience.  However,  during  conduct  of  the  exercise,  the  brigade  staff  leaders  included 
as  much  of  their  staff  as  possible.  This  resulted  in  a  focus  on  the  brigade  staff 
sections  rather  than  just  the  section  leaders.  Additionally,  some  of  the  battalions 
leaders  requested  observation  and  feedback,  even  though  the  TSP  contained  materials 
for  observation  and  feedback  for  the  brigade  staff  only. 

•  Another  lesson  learned  from  the  BSE  trials  was  that  the  BSE  brigade  organization, 
including  four  battalions  and  a  cavalry  troop,  was  not  acceptable  to  the  trial  units.  It 
simply  did  not  replicate  their  normal  brigade  structure.  As  a  result,  the  fourth 
battalion  and  the  cavalry  troop  were  “written  out”  of  the  scenario  and  the  simulation 
materials.  This  was  accomplished  by  the  trial  units  and  their  simulations  center 
personnel,  prior  to  the  exercise  trials. 

•  One  more  lesson  learned  from  the  last  COBRAS  II  trial  was  that  units  wanted 
products  to  help  them  prepare  for  NTC  rotations.  They  consistently  expressed  a  need 
for  additional  training  exercises  that  would  include  24-hour  operations  instead  of  the 
10-12  hour  training  days  built  into  the  BSE,  deployed  command  posts  (CPs),  and 
concurrent  missions. 

These  lessons  from  the  first  two  exercise  trials  were  pivotal  in  defining  the  BBSE 
requirements.  Additional  lessons  learned  were  derived  from  a  third  trial,  conducted  in  May 
1997,  at  Fort  Lewis  (3  Brigade  [Bde],  2  Infantry  Division  [ID]).  In  this  implementation,  the 
BSE  was  being  used  as  an  NTC  ramp-up  exercise.  Among  the  modifications  made  to  suit  this 
purpose  were  changes  in  task  organization  (use  of  one  armor,  one  mechanized  infantry,  and  one 
light  infantry  battalion),  schedule  (24-hour  operations  and  overlapping  missions  and  concurrent 
planning  and  operations,  rather  than  sequential  missions),  and  opposing  force  (OPFOR)  activity 
and  aggressiveness.  All  of  these  changes  were  intended  to  provide  a  virtual  replication  of  the 
anticipated  NTC  experience. 

The  BSE  had  been  intended  as  a  “crawl-level”  exercise,  to  help  brigade  staff  members 
learn  about  their  own  jobs  within  the  larger  staff  process,  to  allow  them  to  practice  interactions 
and  information  flow,  and  to  give  them  experience  in  using  all  of  their  assets  -  combat,  CS,  and 
CSS.  The  May  implementation  was  an  attempt  to  adapt  this  exercise  as  a  “graduate-level” 
exercise,  preparatory  to  their  NTC  rotation.  More  than  words,  these  actions  expressed  the 
brigade’s  need  for  simulation-based  practice  opportunities  that  would  help  to  prepare  them  for  a 
high-intensity,  realistic  field  exercise  and,  by  extension,  for  a  real  world  mission-required 
deployment. 

These  lessons  from  various  implementations  led  to  the  following  four  characteristics  of 
the  BBSE: 

•  The  BBSE  focuses  on  the  commanders,  staff  members,  and  staff  sections  at  both  the 
brigade  and  battalion  levels. 
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•  The  BBSE  has  three  maneuver  battalions  (two  armor  and  one  mechanized  infantry) 
and  does  not  include  a  cavalry  troop  in  its  task  organization.  All  other  brigade  slice 
elements  are  similar  to  the  BSE. 

•  The  BBSE  accommodates  24-hour  operations  and  requires  concurrent  actions  of 
future  mission  planning  and  current  operations  of  different,  unrelated  missions. 

•  The  BBSE  enemy  force  is  allowed  to  be  more  aggressive  and  audacious,  within 
limits,  imposing  a  greater  challenge  on  the  participating  unit. 

COBRAS  III  Program  Design  Synopsis 

As  in  the  COBRAS  I/H  BSE,  the  COBRAS  EH  BBSE  was  designed  according  to  the 
principles  of  structured  training.  These  principles  include:  (a)  a  focus  on  selected  performance 
objectives,  (b)  standardized  exercise  control  to  cue  performance,  (c)  standardized  feedback  to 
correct  and  reinforce  performance,  and  (d)  exercise  support  by  means  of  a  comprehensive  TSP. 
The  basic  methodology  for  design  and  development  is  described  briefly  in  Appendix  C. 

The  following  synopsis  provides  a  brief  overview  of  the  exercises  and  materials  to  be 
developed.  It  describes  the  exercise  design  features,  the  characteristics  of  the  program’s  tactical 
scenario,  the  program’s  instructional  design  characteristics,  and  the  characteristics  of  the  TSPs. 

Exercise  Design  Features 

The  SOW  called  for  development  of  an  exercise  with  three  missions  similar  to  those  the 
BSE  created  in  the  COBRAS  II  effort.  However,  the  new  exercise  is  differentiated  by  the  scope 
of  the  focus. 

•  The  BBSE  is  a  large  scale,  multiechelon  training  exercise  that  immerses  the  key 
brigade  combat  team  members  (i.e.,  brigade  commander,  his  staff,  staff  sections,  and 
one  or  more  of  the  maneuver  battalion  commanders  and  their  staffs)  in  a  multi¬ 
mission  scenario  covering  all  phases  of  the  missions  (plan,  prepare,  and  execute, 
including  consolidation  and  reorganization). 

•  The  exercise  overlaps  the  missions,  forcing  the  BCT  to  plan  for  one  mission  while 
preparing  for  and  executing  another  mission.  This  design  feature  also  has  the  effect 
of  precluding  multiple  entry  points:  Training  using  the  BBSE  must  begin  with  the 
deliberate  attack  (DATK)  and  then  continue  with  the  movement  to  contact  (MTC)  and 
area  defense  (AD).  However,  the  brigade  commander  may  elect  to  end  the  exercise 
after  one,  two,  or  all  three  missions. 

•  The  exercise  requires  the  brigade  combat  team  (BCT)  to  conduct  24-hour  operations 
over  a  5-6  day  period,  requiring  them  to  work  in  shifts. 

•  The  exercise  induces  flexibility  by  changing  the  scope  of  a  mission  or  selected 
conditions  during  the  planning  or  preparation  phase.  This  causes  the  BCT  to  reassess 
its  plan,  make  adjustments,  and  disseminate  the  revisions  in  a  timely  manner. 

•  The  exercise  focuses  on  performance  objectives  for  the  combined  audience  of 
commander  and  staff  members  rather  than  on  discrete  or  individual  tasks. 
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Tactical  Scenario 

The  exercise  is  based  on  multiple  scenarios  that  are  linked  by  the  results  of  the  brigade’s 
operational  status.  In  the  scenario,  a  conventional  mounted  brigade  is  opposed  by  a  Krasnovian 
enemy  operating  according  to  the  conventional  Soviet-style  threat  specifications  described  in  the 
U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Pamphlet  (PAM)  350-16  (Department 
of  the  Army  [DA],  1994). 

The  brigade  faces  a  new  OPFOR  for  each  mission,  yet  is  required  to  perform  the  required 
CSS  actions  to  bring  their  own  forces  back  to  an  operational  strength  needed  to  defeat  the 
OPFOR  in  each  mission.  The  scenario  is  set  on  NTC  terrain  and  covers  the  plan,  prepare,  and 
execute  phases  of  each  mission,  with  a  focus  on  CS  and  CSS  operations. 

Instructional  Design 

The  instructional  design  of  the  BBSE  was  to  be  consistent  with  the  design  of  the  initial 
BSE,  while  accommodating  both  the  expanded  training  audience  and  the  lessons  learned  in 
COBRAS  m.  Like  the  BSE,  the  BBSE  was  to  be  exportable  and  implementable  without  the 
benefit  of  a  dedicated  observer/controller  (O/C)  team.  This  meant  that  all  participants,  including 
O/Cs,  would  have  to  come  from  within  the  training  brigade  or  its  division,  or  from  a  sister 
brigade.  It  also  meant  that  the  TSP  would  be  completely  self-contained,  requiring  no  contractor 
support  team  to  explain  how  to  implement  the  program. 

The  greatest  change  in  the  BBSE  from  an  instructional  standpoint,  compared  to  the  BSE, 
is  that  the  training  objectives  focus  would  be  on  collective  or  team  activities  that  were 
multiechelon  and  that  crossed  battlefield  functional  areas  (BFAs).  These  performance  objectives 
were  identified  and  selected  through  examination  of  experiences  from  the  CTCs  and  from  review 
of  relevant  Army  literature.  The  focus  was  instrumental  in  ensuring  that  the  activities  and 
feedback  for  the  full  multiechelon  BCT  training  audience  were  integrated  throughout  the 
exercise. 

Training  Support  Packages 

The  development  of  the  TSP  was  to  start  with  the  materials  developed  for  the  BSE.  The 
model  for  the  BSE  included  a  single  TSP  that  would  be  flexible  enough  to  be  used  under 
different  implementation  configurations. 

The  TSP  for  the  BBSE  would  need  to  accommodate  the  five  unique  exercise  design 
features  described  above,  as  follows: 

•  Because  of  the  expanded  training  audience,  additional  observation  and  feedback 
materials  would  be  needed.  Additionally,  the  addition  of  the  battalion  commander(s) 
and  staff  to  the  training  audience  meant  that  they  would  operate  from  a  deployed  CP 
rather  than  from  a  Brigade/Battalion  Battle  Simulation  (BBS)  workstation.  Therefore, 
workstation  materials  would  be  required  for  the  company/team  personnel  who  would 
be  at  the  workstation. 

•  The  inclusion  of  overlapping  missions  caused  developers  to  devise  methods  for 
presenting  messages  and  reports  for  both  missions,  while  clearly  indicating  the 
particular  mission  to  which  the  information  applied.  For  example,  during  the  DATK 
mission,  there  would  be  reports  of  enemy  activity  as  they  prepared  their  defense. 
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Simultaneously,  reports  of  enemy  actions,  preparatory  to  their  attack,  would  be 
arriving  to  paint  the  picture  for  the  AD  mission.  Because  the  two  enemy  situations 
were  completely  independent,  it  was  necessary  to  distinguish  between  the  two 
conflicting  sets  for  reports  to  avoid  confusion  in  the  brigade  operations  and  planning 
cells. 

•  The  change  to  a  24-hour  training  day  required  forethought  and  planning  for  staffing, 
timing  of  scenario  events,  and  scheduling  of  feedback  sessions.  In  almost  every 
element  of  the  support  requirement  (e.g.,  roleplayers,  interactors),  staffing 
requirements  would  be  almost  doubled.  The  scenario  timeline  was  constructed  so  as 
to  ensure  that  most  (but  not  all)  of  the  significant  events  would  occur  while  the 
brigade  and  battalion  primary  staffs  were  in  their  CPs,  rather  than  when  the  second 
shift  staffs  were  there.  Finally,  because  the  exercise  tempo  was  already  intense,  the 
feedback  sessions  were  reduced  to  one  per  day,  and  were  structured  to  be  very  austere, 
requiring  less  time  than  multimedia  after  action  reviews  (AARs)  do. 

•  The  introduction  of  flex  missions  to  the  unit’s  scenario  and  the  provision  that  the 
OPFOR  could  exercise  any  of  several  courses  of  action  meant  that  the  TSP  materials 
for  controlling  the  scenario  storyline  had  to  be  written  as  guidance,  with  options  and 
considerations,  rather  than  as  pure  instruction.  Even  in  the  BSE,  there  was  some 
flexibility  given  in  the  different  courses  that  the  scenario  could  follow,  but  it  was 
relatively  slight,  compared  to  the  BBSE. 

•  The  focus  on  multiechelon,  multifunction  performance  objectives  led  to  a  variety  of 
changes  to  the  structure  of  the  observation  and  feedback  materials.  Observers  were  to 
be  organized  to  focus  on  performance  objectives,  rather  than  on  individual  staff 
members  or  single  BFAs. 


Summary 

The  COBRAS  DI  BBSE  design  and  development  process  was  a  carefully  considered 
extension  of  COBRAS  I/n  work.  It  was  never  intended  to  replace  the  BSE,  but  rather  to  provide 
another  exercise  and  TSP  in  the  arsenal  for  brigade  training.  In  the  following  section,  the 
description  of  the  COBRAS  HI  development  plan  demonstrates  clearly  the  similarities  between 
the  projects,  in  terms  of  the  development  methodology  employed  and  the  evaluation  plan. 
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Section  2:  Brigade  and  Battalion  Staff  Exercise  Development  and  Evaluation  Plan 

Like  its  predecessors  COBRAS  HI  followed  the  documented  procedures  for  the 
development  of  structured  simulation-based  training  (C.  H.  Campbell  &  Deter,  1997; 

C.  H.  Campbell,  Deter,  &  Quinkert,  1997).  Consistent  with  the  procedures  and  the  project  SOW, 
the  conduct  of  a  formative  evaluation  was  a  vital  aspect  of  the  COBRAS  in  effort.  The 
evaluation  strategy  required  pilot  implementations  and  continual  reviews  of  the  exercise 
throughout  the  course  of  its  development.  The  findings  from  the  evaluation  were  key  to 
improving  the  program  in  terms  of  its  performance  focus  and  implementation  approach. 

This  section  of  the  report  describes  the  development  method  and  timeline  of  the 
COBRAS  HI  project.  The  three  major  topics  include: 

•  Development  Procedures :  This  section  is  an  overview  of  the  procedures  in  exercise 
development. 

•  Formative  Evaluation  Approach:  This  section  describes  the  evaluation  approach  and 
provides  a  brief  introduction  to  the  evaluation  activities. 

•  Development  Process :  This  section  describes  the  major  project  events  and  phases. 

Development  Procedures 

The  development  process  detailed  for  the  COBRAS  I/II  work  (C.  H.  Campbell  et  al., 
1998)  was  to  drive  the  COBRAS  EH  development  of  the  BBSE.  This  process  includes  a  four- 
phase  set  of  activities.  It  begins  with  complete  delineation  of  all  constraints  and  expectations  for 
the  product,  including  the  intended  training  audience,  simulation,  missions,  and  implementation 
model  considerations. 

The  second  and  third  phases  in  development  are  usually  performed  in  concert.  One  set  of 
procedures  involves  the  specification  of  the  training  objectives  for  the  exercise,  in  terms  of 
specific  individual  or  collective  tasks.  The  other  parallel  activity  is  the  development  of  scenario 
conditions,  including  task  organization  and  modified  table  of  organization  and  equipment 
(MTOE);  starting  locations,  dispositions,  and  readiness  levels;  enemy  intent  and  order  of  battle; 
higher  echelon  missions;  and  a  scenario  storyline  defining  how  the  exercise  events  will  unfold. 
Activities  in  both  phases  are  guided  and  defined  by  the  decisions  documented  in  the  first  phase. 

The  fourth  phase  in  development  involves  the  design  and  development  of  the  TSP  itself, 
incorporating  all  of  the  products  from  the  second  and  third  phases.  The  TSP  architecture  must 
answer  all  of  the  constraints  laid  out  in  the  first  phase  as  well. 

Because  the  COBRAS  HI  BBSE  represented  an  expansion  of  the  COBRAS  I/H  BSE,  a 
modified  application  of  the  methodology  was  employed.  This  modification  enabled  developers 
to  use  many  of  the  exercise  components  already  developed  (e.g.,  scenario,  performance  job  aids, 
scripted  messages).  Each  component  was  reviewed  and  modified  as  necessary  before  being 
added  to  the  BBSE  TSP.  The  full  implementation  of  the  development  process  is  detailed  in 
Section  3  of  this  report. 
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Formative  Evaluation  Approach 

An  integral  and  critical  aspect  of  development  was  the  formative  evaluation.  In  the 
context  of  structured  training  development,  “formative  evaluation”  means  an  ongoing  cycle  of 
examine-evaluate-refine.  Formative  evaluation  occurs  not  after  a  project’s  completion,  but 
rather,  during  the  development  and  initial  implementations  of  the  program. 

The  evaluation  method  for  the  COBRAS  III  program  was  modeled  on  the  process 
followed  for  COBRAS  I  and  COBRAS  II  (Graves  et  al.,  1997;  C.  H.  Campbell  et  al.,  1998).  The 
examine-evaluate-refine  cycle  was  executed  throughout  every  phase  of  design  and  production  to 
ensure  that  each  component  of  the  BBSE  and  vignettes  was  functional,  doctrinally  correct,  and 
consistent  with  other  components. 

The  BBSE  formative  evaluation  began  as  the  project  itself  commenced,  with  assessment 
of  how  to  implement  the  initial  design  decisions,  consistent  with  and  in  support  of  the  project 
objectives.  For  example,  with  the  training  audience  was  now  expanded  to  include  both  the 
brigade  staff  and  the  battalion  staff.  The  performance  objectives  were  to  be  primarily,  if  not 
totally,  multiechelon,  addressing  both  levels  within  each  objective.  As  additional  decisions  were 
made  and  materials  developed,  further  assessments  were  made  regarding  whether  these  would 
also  support  the  project  objectives.  The  early  evaluation  activities  were  primarily  internal  to  ARI 
and  the  development  team,  with  briefings  and  guidance  from  the  Force  XXI  Training  Program 
and  the  USAARMC  at  critical  decision  points. 

As  BBSE  development  efforts  proceeded,  evaluation  activities  became  more  formal  and 
comprehensive  in  nature  and  increasingly  significant  external  support  was  required.  These 
included  both  exercise  pilot  runs  and  trial  implementations.  The  evaluation  activities  are 
described  briefly  below  with  details  of  the  evaluation  conduct  and  results  in  Section  4. 

For  the  BBSE,  pilot  exercises  were  conducted  with  program  developers  assisted  by 
developers  of  other  structured  training  programs  and  simulation  site  staff.  During  these  pilot 
tests,  the  focus  was  on  assessment  of  the  correctness  and  usability  of  selected  components  of  the 
exercises.  For  example,  pilot  tests  were  conducted  to  ensure  that  the  content  and  timing  of 
scripted  messages  from  the  roleplayed  division  would  be  appropriate  and  that  enemy  movements 
would  result  in  engagements  as  expected. 

The  trial  implementation  involved  users  representative  of  the  intended  training 
participants  and  was  conducted  to  allow  evaluation  of  all  components  of  the  TSP.  It  provided  the 
forum  for  obtaining  comprehensive  user  input  regarding  efficacy  of  design  decisions,  material 
functionality,  user  acceptability,  and  perceived  training  benefit.  For  the  BBSE,  a  full  trial 
implementation  was  conducted  in  the  3  Bde,  2  ID,  of  Fort  Lewis,  Washington. 

The  methods  employed  to  collect  information  changed  between  the  early  informal 
evaluation  processes  and  the  pilots  and  trials.  During  early  stages  of  development,  the  activities 
were  relatively  unstructured,  and  very  little  systematic  data  collection  and  analysis,  in  a 
traditional  sense,  was  performed.  Developers  focused  their  attention  on  specific  processes  that 
they  needed  to  assess.  For  example,  in  early  simulation-supported  tryouts  of  the  scenario 
settings,  the  scenario  contained  initial  conceptions  of  events  that  were  to  occur  to  drive  the 
activities  of  the  newly  added  battalion  training  audience  personnel.  The  evaluations  were  aimed 
at  determining  if  the  simulation  representation  of  the  scenario  would  function  as  intended, 
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providing  cues  and  conditions  as  designed.  A  scenario  might  be  tested,  revised,  and  tested  again 
in  the  same  day;  consequently,  the  focus  was  on  obtaining  and  applying  information  rapidly. 

During  the  later  pilot  tests,  the  data  collection  was  more  systematic  and  specific. 
Expected  outcomes  were  targeted  before  each  pilot  or  trial,  in  the  form  of  specific  questions,  and 
evaluation  instruments  were  designed  to  facilitate  the  collection  of  information  to  address  the 
issues.  Generally,  pilot  results  included  corrections,  clarifications,  resolution  of  inconsistencies, 
amplifications  or  elaboration,  deletions,  and  development  of  new  materials  and  procedures  as 
solutions  to  specific  design  problems. 

The  trial  implementation  was  larger  in  scope  than  the  pilots  and  consequently  involved 
more  extensive  formative  evaluation  procedures.  The  formative  evaluation  instruments  for  the 
trial  were  designed  to  support  both  observational  and  querying  methods  of  data  collection.  The 
instruments  generally  included  structured  interview  guides,  evaluator  guides,  questionnaires,  and 
group  discussion  guides.4 

Each  evaluation  exercise  was  designed  for  timely  documentation  of  results  so  that 
information  could  be  disseminated  quickly  among  the  training  developers.  Involvement  of  the 
training  developers  in  the  observation  process  facilitated  the  extraction  of  findings  and  their 
translation  into  steps  for  modification.  The  observations  and  data  from  each  exercise  were 
organized  and  analyzed  rapidly.  The  findings  were  then  distributed  immediately  to  the  training 
developers  and  dialog  sessions  were  organized,  as  necessary,  to  clarity  and  prioritize  revision 
actions. 


Development  Process 

As  described  above,  the  project  followed  the  procedure  for  development  of  structured 
simulation-based  training.  It  is  not  a  strictly  linear  procedure,  but  rather  anticipates  considerable 
modifications  to  design  decisions  as  further  development  continues. 

The  development  of  the  BBSE  scenario,  identification  of  the  performance  objectives,  and 
construction  of  the  TSP  began  just  after  the  COBRAS  II  BSE  was  implemented  at  Fort  Lewis 
(3  Bde,  2  ID)  in  May  1997.  That  exercise,  as  described  earlier,  was  conducted  with  some 
significant  modifications  to  the  scenario  and  implementation  model  in  order  to  provide  a  virtual 
replication  of  the  anticipated  NTC  experience. 

The  general  goal  and  intent  for  the  BBSE  arose  out  of  that  trial  implementation.  As 
discussed  in  Section  3,  the  changes  instituted  by  the  unit  formed  the  basis  of  the  initial  decisions 
for  the  BBSE.  The  design  processes  of  performance  analysis  and  scenario  construction  were 
overlaid  with  TSP  construction  in  order  to  meet  a  BBSE  trial  implementation  set  to  occur  in 
November  1997. 

The  trial  implementation  of  the  BBSE,  described  in  Section  4,  occurred  as  planned  at  Fort 
Lewis,  Washington,  in  November  1997.  Participants  included  members  of  the  3  Bde,  2  ID  and 
the  181  Brigade  Reserve  Training  Detachment.  The  trial  enabled  developers  to  try  out  and  obtain 
information  on  many  aspects  of  the  exercise,  including  the  performance  objectives  and  exercise 


4  Questionnaires  were  reviewed  by  ARI  prior  to  the  trial  to  ensure  rights  of  the  respondents  would  be  protected.  The 
questionnaires  were  assigned  ARI  Personnel  Test  Number  60-02. 
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timelines.  After  the  trial,  intense  internal  reviews  of  the  TSP  were  conducted,  and  data  were 
analyzed  and  applied  to  improve  the  exercise. 

Throughout  the  project,  Force  XXI  Training  Program  and  USAARMC  were  kept 
informed  of  the  progress  and  obstacles  by  means  of  a  series  of  in-progress  reviews. 

Summary 

Within  the  development  and  evaluation  plan  described  here,  there  were  various 
considerations  and  decisions  that  shaped  the  final  products.  The  next  two  sections  of  the  report 
describe  the  design  and  development  events.  Section  3  describes  the  production  of  the  BBSE 
and  Section  4  describes  the  trial  implementation  and  evaluation  findings.  Following  the 
development-oriented  sections,  the  report  provides  lessons  learned  and  a  description  of  the  needs 
that  remain  to  be  addressed  by  future  R&D. 
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Section  3:  Development  of  the  Brigade  and  Battalion  Staff  Exercise 

Because  the  developmental  process  for  COBRAS  ID  is  very  similar  to  the  COBRAS  I/n 
process  (Graves  et  al.,  1997;  C.H.  Campbell  et  al.,  1998),  this  report  highlights  the  aspects  of 
development  and  the  exercise  itself  that  were  unique  to  the  COBRAS  HI  effort.  It  identifies 
those  aspects  of  the  BBSE  that  were  changed  during  the  project,  how  they  were  changed,  and  the 
outcomes  of  those  changes  (i.e.,  new  or  revised  TSP  components). 

This  section  deals  with  BBSE  development  in  preparation  for  the  trial  implementation  at 
Fort  Lewis,  Washington  in  December  1997.  The  topics  covered  include; 

•  Designating  the  Primary  Training  Audience:  This  section  describes  how  the  SOW 
requirement  evolved  into  designation  of  the  specific  training  audience  members. 

•  Designing  the  Tactical  Scenario:  This  section  explains  how  the  COBRAS  I/H 
scenario  was  altered  to  include  scenario  events  that  would  cue  the  involvement  of  the 
brigade  and  battalion  training  audience  members,  to  allow  for  concurrent  planning, 
and  to  provide  conditions  for  practicing  complex  performance  objectives. 

•  Designating  the  Performance  Objectives:  This  section  describes  the  process  for 
identifying  performance  objectives  and  constructing  the  supporting  documentation, 
task  identification  and  selection  process  and  its  products. 

•  Designing  the  Exercise  Architecture:  This  section  presents  the  architecture  of  the 
COBRAS  HI  BBSE  as  it  was  prepared  for  the  trial,  and  how  it  differed  from  that  of 
the  BSE. 

•  Developing  the  Training  Support  Package:  This  section  provides  a  synopsis  of  the 
BBSE  TSP  as  it  was  prepared  for  the  trial.  Special  focus  is  placed  on  describing  how 
the  COBRAS  HI  TSP  differed  from  earlier  versions. 

Designating  the  Primary  Training  Audience 

The  first  and  most  consequential  decision  in  designing  the  BBSE  was  identifying  new 
training  audience  members.  The  initial  decision  made  by  ARI  and  the  COBRAS  developers  to 
include  brigade  staff  section  members  and  the  commander  and  staff  of  one  or  more  maneuver 
battalions  in  the  primary  training  audience  was  directive  and  clear,  but  not  very  specific. 
Attempts  to  enumerate  exactly  who  would  be  considered  “primary  training  audience”  were 
largely  unsuccessful  and,  on  reflection,  unnecessary.  Precise  specifications  of  the  training 
audience  individuals  were  important  in  the  BSE,  because  the  exercise  objectives  were  written  in 
terms  of  the  roles  of  individuals  within  staff  processes.  However,  for  the  BBSE,  with  the  focus 
more  broadly  on  collective  skills  and  performance,  the  commanders  and  staff  leaders  would  be 
expected  to  determine  who,  within  the  staff  sections,  should  participate.  Their  further  decision, 
concerning  who  would  attend  the  large  group  feedback  sessions,  would  operationally  define  the 
training  audience.  The  performance  objectives  would  be  written  to  the  BCT  and  the  specific 
activities,  rather  than  to  individuals  within  a  group  process. 

Representatives  of  the  Force  XXI  Training  Program  and  the  USAARMC  concurred  with 
the  definition  of  the  training  audience  members  as  well  as  the  rationale  for  this  decision.  This 
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was  an  important  step  in  ensuring  that  the  expectations  of  the  proponent  agencies  and  program 
stakeholders  would  be  met. 


Designing  the  Tactical  Scenario 

The  BBSE  scenario  was  not  developed  from  scratch,  but  represented  a  revision  of  the 
COBRAS  I/n  scenario.  In  the  early  stages  of  scenario  redesign,  efforts  were  driven  by  four 
distinct  considerations: 

1.  to  design  a  storyline  that  would  allow  for  concurrent  planning  of  a  follow-on  mission 
while  still  preparing  for  and  executing  the  prior  mission, 

2.  to  ensure  that  storyline  would  not  put  the  most  critical  scenario  events  during  the 
hours  when  second  shift  staff  members  would  be  operating  the  CPs, 

3.  to  provide  options  for  alternative  plans  and  execution  for  the  OPFOR  activities, 

4.  to  incorporate  the  task  organization  changes  recommended  after  the  BSE  trials  (i.e., 
using  only  three  battalions  and  no  cavalry  troop). 

The  goal  of  providing  for  concurrent  missions  was  particularly  difficult  to  achieve. 
Because  the  BCT  could  select  any  one  of  a  theoretically  infinite  number  of  courses  of  action  and 
could  execute  its  course  of  action  in  any  number  of  ways,  it  was  impossible  to  predict  where  and 
in  what  condition  it  would  be  when  the  transition  to  the  follow-on  mission  occurred.  In  order  to 
force  CSS  activities,  developers  wanted  to  allow  the  brigade  to  rearm  and  resupply,  and  needed 
to  allow  time  for  that  to  happen.  Additionally,  the  brigade  would  need  to  be  in  a  specific  starting 
location  for  the  subsequent  mission  in  order  for  developers  to  prepare  simulation  files  for  the 
OPFOR.  Therefore  the  decision  was  made  to  construct  a  storyline  that  had  the  brigade  return  to 
the  assembly  area  between  missions,  required  the  brigade  to  carry  out  sustainment  activities  in 
order  to  get  to  a  sufficient  level  of  readiness,  and  included  a  “new”  OPFOR  that  was  not  depleted 
for  each  mission.  As  a  result,  the  storyline  is  not  strictly  continuous  for  the  three  missions. 

The  second  and  third  considerations  listed  above  were  not  particularly  difficult  to 
accommodate.  They  required  attention  and  adjustment,  but  did  not  involve  any  real  innovation 
in  design. 

Work  to  integrate  the  adjusted  storyline  into  the  scenario  began  with  the  revision  of 
tactical  products,  starting  with  the  corps  and  division  operations  order  (OPORD)  materials. 

Other  aspects  of  the  scenario,  such  as  products  that  contained  ancillary  information  (e.g., 
personnel  status,  levels  of  supply  and  maintenance,  intelligence  summary  information, 
meteorological  and  terrain  information)  that  would  flow  down  from  division  and  corps,  were  also 
altered  to  provide  performance  cues  as  necessary.  Together,  these  products  were  the  structuring 
mechanism  of  the  scenario,  as  they  set  the  situation  and  conditions  under  which  the  BCT  would 
develop  its  plan. 

The  initial  testing  of  the  scenario  occurred  in  map  exercises  conducted  to  ensure  that  the 
scenario  was  tactically  sound  and  that  the  actions  required  of  the  BCT  were  consistent  with 
current  doctrine  as  well  as  internally  consistent.  The  scripted  messages  received  special  scrutiny 
to  ensure  that  the  BCT  would  be  able  to  distinguish  the  messages  for  the  current  mission  from 
those  for  the  subsequent  mission. 
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Following  the  initial  map  exercises,  and  when  the  tactical  products  were  judged  as  being 
sufficiently  complete,  simulation-based  tryouts  were  conducted,  much  like  those  conducted 
during  the  BSE  development.  The  tryouts  required  the  development  of  BBS  electronic  files, 
which  contained  the  details  of  the  task  organization,  readiness  levels,  and  operational  graphics. 
Their  purpose  was  to  ensure  that  the  scenario  would  unfold  as  designed,  producing  the  desired 
events  and  cues. 

Mission  by  mission,  the  scenario  was  evaluated  for  its  capacity  to  logically  and 
realistically  contain  the  events  that  would  cue  the  intended  performance  requirements.  The 
resulting  scenario  and  tactical  products  were  judged  to  be  appropriate  to  support  continuing 
development. 

Table  1  contains  a  brief  description  of  the  COBRAS  III  BBSE  scenario  storyline. 

Table  1 

Summary  of  COBRAS  III  Scenario  Storyline 
The  Road  to  War 

The  scenario  storyline  begins  with  the  brigade  in  a  field  training  exercise  (FTX),  having  been 
deployed  to  the  country  of  Mojave  because  of  an  increasing  threat  from  the  Krasnovians.  The 
FTX,  which  creates  decrements  in  personnel  and  equipment  status,  ends  as  intelligence  indicates 
that  the  Krasnovians  are  preparing  to  cross  the  international  border. 

The  Area  Defense  (AD) 

The  brigade  staff  then  issue  an  order  for  an  AD  mission  and  must  simultaneously  conduct  its 
consolidation  and  reorganization  activities  and  plan  for  the  AD.  The  Krasnovians  conduct  their 
attack,  which  fails  after  inflicting  some  losses  to  the  brigade.  The  brigade  then  conducts  a 
rearward  passage  of  lines  to  a  rear  assembly  area  ([AA];  the  passage  of  lines  is  not  performed  in 
the  exercise). 

The  Deliberate  Attack  (DATK) 

Three  days  later  (these  three  days  are  not  represented  in  the  exercise;  in  fact,  the  division  order  is 
issued  before  the  AD  is  executed),  the  brigade  receives  a  subsequent  order  to  conduct  a  DATK 
mission  against  the  remnants  of  the  Krasnovians.  The  three  days  in  the  scenario  provide  a  realistic 
time  for  the  brigade  to  return  its  forces  to  a  level  of  combat  readiness,  which  will  make  the  DATK 
mission  feasible.  Again,  the  brigade  staff  develops  its  plan,  briefs  the  plan,  monitors  preparation 
activities,  and  conducts  the  attack.  The  storyline  ends  with  the  defeat  of  the  Krasnovians  and 
return  to  the  AA. 

The  Movement  to  Contact  (MTC) 

While  the  brigade  is  in  its  AA,  conducting  medical,  repair,  and  replacement  activities,  it  receives 
an  order  to  conduct  a  MTC  mission  (in  the  exercise,  the  division  order  is  issued  before  the  DATK 
is  executed.  The  brigade  staff  prepares  their  order,  while  subordinate  units  continue  their  logistics 
efforts.  The  MTC  is  conducted  against  the  Krasnovians’  meeting  battle.  After  the  brigade  has  met 
and  defeated  the  Advance  Guard  Main  Body,  circumstances  require  the  brigade  and  the  enemy  to 
stop  their  advances  and  take  up  hasty  defenses.  The  Krasnovian  lead  elements  retreat  to  hasty 
defensive  positions  awaiting  the  arrival  of  second  echelon  forces  to  conduct  an  attack. _ 
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Designating  the  Performance  Objectives 

Discussions  with  the  3  Bde,  2  ID  during  the  May  1997  implementation  and  subsequent 
discussions  following  the  brigade’s  NTC  rotation  led  to  the  conceptualization  of  performance 
objectives  that  were  more  inclusive  and  less  procedural  than  tasks,  even  collective  tasks.  An 
especially  clear  example  of  the  needed  objectives  came  during  one  of  the  AARs  at  the  NTC, 
when  the  O/C  was  reviewing  a  broad  sweep  of  fire  support  issues  with  the  commander  and  the 
brigade  and  battalion  members.  Not  only  was  it  clear  that  this  multi-battlefield  operating  system 
(BOS)  approach  could  be  applied  for  a  variety  of  brigade  requirements,  but  it  was  also  apparent 
that  a  simulation-based  exercise  could  provide  opportunities  to  practice  techniques  and 
procedures  that  would  give  the  staff  an  edge  in  performing  key  high-payoff  functions. 

For  example,  one  such  focus  could  be  on  how  the  brigade  and  battalion  CPs  manage 
information  flow  throughout  all  phases  of  a  mission.  Another  focus  could  be  on  the  planning  and 
management  of  the  reconnaissance  and  surveillance  (R&S)  for  the  brigade.  Yet  another  could 
concern  how  the  brigade  and  battalions  manage  their  parallel  planning  efforts.  By  practicing 
these  objectives  prior  to  their  CTC  rotation,  the  BCT  would  be  better  ready  to  take  advantage  of 
the  O/C  expertise,  challenging  scenarios,  and  live  conditions. 

Because  of  the  multiechelon  emphasis  for  the  BBSE,  multiechelon  performance 
objectives  were  required.  Early  development  included  both  multiechelon  performance  objectives 
and  single-level  objectives,  as  shown  in  Figure  1 .5  These  were  the  objectives  that  were  used 
during  the  trial  implementation  of  the  BBSE  in  November  1997. 


Plan  and  Implement  Brigade  Air  Defense  Early  Warning 
Plan  and  Execute  Engagement  Areas 
Parallel  Planning  within  the  Brigade 
Manage  Information  within  the  Brigade  Command  Posts 
Manage  Information  within  the  Task  Force  Command  Posts 
Plan  and  Manage  Reconnaissance  within  the  Brigade 
Integrate  CSS  Estimates  in  Decision-Making 
Conduct  Clearance  of  Indirect  Fires  Procedures 

Figure  1.  Brigade  and  Battalion  Staff  Exercise  performance  objectives  for  Fort  Lewis  trial  in 

November  1997. 


The  materials  for  the  training  audience  and  the  observers  were  derived  from  a 
combination  of  subject  matter  expertise  within  the  COBRAS  team  and  Army  personnel,  and 


5  A  complete  description  of  the  development  process  for  performance  objectives  is  provided  in  Jenkins,  Graves, 
Deter,  &  Quinkert  (in  preparation).  This  ARI  Research  Product  also  contains  the  performance  objectives 
materials  developed  for  the  BBSE. 
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experiences  of  combat  veterans  and  CTC  observers  (especially  as  documented  by  the  Center  for 
Army  Lessons  Learned  [CALL]  at  Fort  Leavenworth). 

Each  performance  objective  description  had  five  key  elements: 

•  Persuasive  statement  of  the  pressing  need  for  expert  performance 

•  Statement  of  the  objective  in  terms  of  the  consequences  of  proficient  performance  or 
flawed  performance 

•  Overview  of  the  performance  requirements 

•  Detailed  techniques  and  procedures 

•  Considerations  for  assessment  for  the  commander  or  the  observer 

•  References  and  related  Mission  Training  Plan  (MTP)  tasks. 

The  techniques  and  procedures  constitute  the  greatest  part  of  each  objective  description. 
They  were  not  written  to  be  prescriptive,  but  rather  to  provide  performance  guidance  for  the 
unit’s  consideration.  They  expanded  on  available  Army  Training  and  Evaluation  Program 
(ARTEP)-MTP  descriptions  by  adding  suggestions  concerning  who  would  perform  what 
essential  parts  of  the  function,  what  products  could  be  useful,  or  how  the  staff  could  provide 
more  timely  support  for  the  commander’s  decision-making. 

The  assessment  guidance  did  not  require  that  the  unit  perform  as  laid  out  in  the 
techniques  and  procedures  section.  Rather,  the  assessment  questions  and  considerations 
addressed  the  objective  statement.  The  unit  could  use  its  own  procedure  or  the  given  procedure; 
the  important  thing  was  that  the  objective  be  accomplished.  Thus  the  techniques  and  procedures 
might  serve  as  guidance  for  one  unit,  but  as  a  checklist  of  considerations  for  another  unit.  The 
three  key  questions  in  assessment  were  “Does  the  unit  have  a  procedure?”,  “Did  the  procedure 
accomplish  the  objective?”,  and,  “Is  the  unit  happy  with  its  procedure,  or  what  should  be 
changed?” 

The  observer  materials  contained  additional  guidance.  Information  was  provided  on 
where  to  observe,  what  to  look  for,  and  what  BBS-generated  data  to  obtain  in  order  to  provide 
feedback  to  the  unit  on  their  processes  and  the  battlefield  effects  of  their  actions.  This  was  not  to 
be  exhaustive  guidance  about  all  aspects  of  the  performance  objective;  the  considerations  for 
assessment  would  provide  most  of  the  observation  guidance.  Rather,  the  observer  guides  would 
detail  BBS-specific  or  BBSE  scenario-specific  suggestions. 

Altogether,  eight  performance  objectives  were  prepared  for  the  November  1997  trial. 

This  was  intended  to  serve  only  as  an  initial  set  to  test  the  concept  and  approach  to  delineating 
the  objectives.  The  goal  was  to  eventually  include  between  12  and  20  such  objectives.  It  seemed 
obvious  that  this  would  be  too  many  for  the  unit  to  focus  on  during  a  5-day  exercise;  even  one 
AAR  per  objective,  at  1-2  hours  per  AAR,  would  use  up  a  significant  portion  of  the  training  time. 
Instead,  the  implementation  model  called  for  the  brigade  commander  to  select  no  more  than  six 
objectives  for  focus.  Observers  would  pay  particular  attention  to  those  selected  objectives  in 
formal  and  informal  feedback  sessions.  All  objectives  would  be  cued,  and  performance  of  all  of 
them  would  (or  should)  occur,  but  the  focus  for  any  given  implementation  would  be  on  a  selected 
set. 
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Designing  the  Exercise  Architecture 

Every  aspect  of  the  original  (COBRAS  I)  BSE  was  precisely  designed  to  support  the 
intent  of  the  exercise,  which  was  driven  by  the  specific  training  audience  and  the  scenario. 
Participants  had  well  defined  and  purposeful  roles,  simulation  layout  and  usage  were  described  in 
specific  terms,  the  scenario  was  highly  structured,  and  the  TSP  was  thorough  and  direct.  Because 
the  exercise  was  not  intended  to  train  all  aspects  of  brigade  functioning,  the  model  specified  8-10 
hour  workdays  and  operations  out  of  administrative  CP  locations  in  or  near  the  simulation  center. 
Thus,  all  components  of  the  design  and  the  TSP  itself  were  created  to  represent  a  tightly  linked 
group  of  products  that,  when  put  together,  would  support  the  functioning  of  all  participants. 

In  the  COBRAS  in  effort,  a  very  different  exercise  model  was  to  be  employed.  The 
differences  were  most  apparent  in  three  areas: 

•  Structure  for  observation  and  feedback 

•  Implementation  conditions 

•  Participant  types  and  numbers. 

Structure  for  Observation  and  Feedback 

Significant  modifications  were  made  for  the  BBSE  design  in  the  area  of  performance 
observation  and  the  provision  of  feedback.  The  role  of  the  observers  in  the  COBRAS  HI  BBSE 
was  essentially  the  same  as  in  COBRAS  I/H:  to  serve  as  mentors,  coaches,  and  providers  of 
feedback,  both  in  informal  discussions  and  in  more  formal  AARs  with  brigade  and  battalion  staff 
members.  But  wherein  the  BSE  observers  were  assigned  to  shadow  one  or  two  members  of  the 
training  audience,  in  the  BBSE,  observers  were  instructed  to  follow  particular  performance 
objectives.  Thus  a  group  of  two  or  three  observers  might  be  tracking  different  aspects  of  R&S 
planning,  and  another  observer  might  be  focused  on  information  flow  in  the  brigade  main  CP. 
The  type  and  number  of  observers  required  would  depend  on  the  particular  performance 
objectives  selected  by  the  commander.  Although  professional  observers,  such  as  those  found  in 
the  regional  training  units,  would  be  best  able  to  learn  and  apply  the  procedures,  the  materials 
were  designed  for  use  by  untrained  observers. 

In  the  BSE,  frequent  AARs  (one  or  two  per  day)  were  used  instead  of  the  more  common 
training  model  of  a  single  AAR  upon  mission  completion.  The  BBSE  compromised  these  two 
approaches.  The  model  called  for  an  AAR  period  of  time  each  evening,  at  about  the  time  that  CP 
shift  changes  would  occur.  During  any  given  AAR,  only  two  or  three  performance  objectives 
would  be  discussed;  each  objective  should  be  the  topic  of  at  least  two  AARs,  so  that  progress  and 
development  could  be  discussed. 

Feedback  would  occur  in  one-on-one  sessions,  small  group  sessions,  or  formal  AARs. 
However,  even  the  most  formal  AARs  would  not  require  audiovisual  aids  or  high  technology 
graphics.  Instead,  the  focus  would  be  on  the  discussion,  with  the  observer  leading  the  brigade 
and  battalion  training  audience  members  to  decide  whether  they  had  a  satisfactory  and  acceptable 
procedure  for  the  objective.  The  intent  was  to  keep  time  expenditures  down  and  to  target  the 
unit’s  performance  and  battlefield  effects  as  efficiently  as  possible. 
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In  any  given  group  feedback  session,  representatives  from  all  BOS  and  both  echelons 
would  be  involved.  The  multiechelon  focus  for  the  full  BCT  was  to  be  maintained  throughout 
the  exercise. 

Implementation  Conditions 

The  conditions  for  implementation  of  the  exercise  were  deliberately  more  challenging 
than  they  had  been  for  the  BSE,  in  keeping  with  the  intent  to  provide  a  training  setting  that  would 
begin  to  approximate  NTC  conditions.  There  were  four  conditions  that  were  noticeably  more 
difficult  for  the  unit: 

•  The  training  schedule  called  for  a  24-hour  5-day  exercise.  From  the  time  of  the 
division  order  being  provided  to  the  brigade,  all  participants  were  fully  involved  with 
a  real-time  simulation  clock  running  continuously.  Real  world  considerations  of 
shifts  and  shift  change  briefings  had  to  be  worked  out  by  the  units. 

•  The  brigade  and  battalion  CPs  were  deployed  adjacent  to  or  close  to  the  BBS 
simulation  facility.  This  increased  the  demands  on  the  unit  to  maintain  their 
communications  equipment  and  procedures,  while  still  allowing  feedback  sessions  to 
be  conducted  without  undue  interruptions. 

•  The  scenario  timeline  presented  the  three  missions  in  an  overlapping  scenario,  rather 
than  sequentially.  This  required  the  units  to  work  on  their  procedures  for  concurrently 
executing  one  mission  while  planning  the  follow-on  mission. 

•  The  OPFOR  was  allowed  more  flexibility  than  in  the  BSE,  and  was  allowed  to  be 
more  aggressive.  Rules  of  engagement  for  the  OPFOR  provided  alternative  courses 
of  action  that  the  Exercise  Director  could  select  in  order  to  provide  the  appropriate 
level  of  challenge  for  the  unit. 

Participant  Types  and  Numbers 

Because  of  the  inclusion  of  one  or  more  battalions  in  the  training  audience,  and  because 
of  the  more  intense  implementation  conditions  described  above,  the  training  audience  and 
requirements  for  additional  participants  increased  greatly.  Specifically: 

•  In  the  BSE,  the  primary  training  audience  was  the  brigade  commander  and  staff  -  a 
total  of  16  persons.  It  also  required  an  austere  number  of  supporting  participants  for 
the  brigade  staff  members,  12  altogether.  For  the  BBSE,  even  with  only  one  battalion 
participating  fully,  the  primary  training  audience  for  two  shifts,  including  minimal 
numbers  of  staff  section  members,  was  169. 

•  Roleplayers  for  the  BSE  portrayed  the  brigade  assets,  and  numbered  32.  Another  44 
individuals  acted  as  interactors  at  the  workstations.  For  the  BBSE,  roleplayers 
portrayed  the  battalion’s  line  companies  and  other  assets  for  each  fully  participating 
battalion.  Additionally,  brigade  asset  roleplayers  had  to  be  sufficient  in  number  to 
task  organize  the  participating  battalion’s  companies,  as  appropriate.  The  roleplayers 
and  interactors  together  numbered  96. 

•  The  number  of  observers  in  the  BBSE  depended  on  the  particular  performance 
objectives  selected,  but  it  was  estimated  that  at  least  20  would  be  required  for  any 
implementation.  Observers  would  not  need  to  be  double  shifted.  Rather,  they  were 
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expected  to  adjust  their  schedules  so  as  to  be  on  site  for  critical  events  in  the  unit’s 
timeline  or  in  the  scenario  timeline. 

•  The  division  response  cell  (known  as  exercise  control  or  EXCON)  and  OPFOR 
changed,  primarily  because  of  the  double  shift  requirement:  a  total  of  14  for  the  BSE 
and  32  for  the  BBSE. 

In  summary,  the  personnel  requirements  increased  from  127  to  317.  Of  this  group,  169 
were  members  of  the  primary  training  audience.  The  remaining  148  constituted  the  supporting 
training  audience  or  observer  team,  all  of  whom  would  derive  significant  training  value  from  the 
exercise. 


Developing  the  Training  Support  Package 

As  the  changes  in  the  participants  and  other  aspects  of  the  BBSE  design  were 
incorporated,  the  BBSE  TSP  also  began  to  be  clearly  different  from  its  predecessor.  The  major 
modifications  apparent  in  the  TSP  prior  to  the  trial  implementation,  compared  to  the  BSE  model, 
included: 

•  The  tactical  materials  were  designed  to  be  overlapping,  as  described  earlier.  This 
required  developers  to  devise  methods  for  clearly  distinguishing  between  the  scripted 
messages  pertinent  to  the  current  mission  and  those  that  applied  to  the  future  mission. 

•  Training  audience  preparation  materials  included  the  performance  objective 
descriptions,  along  with  guidance  that  the  unit  should  use  the  techniques  and 
procedures  to  review  and  refine  their  existing  procedures,  or  could  choose  to  adopt 
the  procedures  provided. 

•  Observer  materials  and  training  audience  materials  contained  fuller  descriptions  of 
possible  types  of  AARs.  The  emphasis  was  on  maintaining  the  low-intrusion  aspect 
of  all  feedback  sessions. 

•  The  roleplayer  and  interactor  guides  were  combined  to  produce  workstation  team 
guides  containing  information  for  both  types  of  workstation  team  member.  This  was 
to  be  the  third  model  tested:  The  COBRAS  I  provided  guides  for  interactors  for 
offense  and  defense,  and  a  separate  guide  for  each  type  of  roleplayer;  COBRAS  II 
provided  combat,  CS,  and  CSS  interactor  guides  for  blue  forces  workstations,  and 
combat  and  CS  for  the  OPFOR  interactors,  as  well  as  separate  roleplayer  guides. 
Developers  continued  their  search  for  a  format  that  would  encourage  use  by 
workstation  team  members  and  Contracted  Logistics  Support  (CLS)  staff. 

Table  2  provides  a  description  of  the  overall  COBRAS  II  BSE  TSP  structure,  as  it  existed 
going  into  the  trial  implementation. 
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Table  2 

Structure  of  the  COBRAS  III  Brigade  and  Battalion  Staff  Exercise  Training  Support 
Package  at  the  Time  of  the  Trial  Implementation 


fort 1 

-r-V  l;r^_  : 

Exercise 

Management 

•  Exercise  Guide  for  the  Exercise  Director,  COBRAS  Coordinator,  and  Blue 
Forces  Controller,  with  Appendixes 

•  Brigade  and  Battalion  Orientation  Guide 

Tactical  Materials 

•  Corps  Concept  (MTC,  AD,  DATK) 

•  Division  Orders  and  Tactical  Materials  (including  overlays) 

•  Scripted  and  hard-copy  messages 

Participant  Guides 
and  Materials 

Training  Audience: 

•  Training  Audience  Guide  (generic,  for  all  training  audience  elements  at 
brigade  and  battalion  levels) 

•  XO  Guide  to  Unit  Preparation  and  Materials  Distribution 

•  Initial  Situation  Packages  and  start  of  exercise  (STARTEX)  Position  Overlays 
(per  staff  member,  per  mission) 

Observers: 

•  Observer  Guide  (generic,  for  all  Observers) 

•  Performance  Objectives 

Workstation  Team  Guides  for: 

•  Exercise  Control  (EXCON)  •  Opposing  Forces 

•  Task  Force  (TF)  1-5  •  Field  Artillery 

•  TF  1-7  •  Engineer 

•  TF  3-5  •  Forward  Support  Battalion  (FSB) 

•  Air  Defense  Artillery  (ADA)  .  Brigade  Troops 

•  Initial  Situation  Packages  and  STARTEX  Position  Overlays  (per  workstation 
team) 

Simulation 

Materials 

Tools  for  initializing  BBS  and  making  changes  or  corrections: 

•  Simulation  Site  Manager’s  Guide 

•  BBS  TOE  and  Initialization  Book 

•  BBS  Archive  Book 

•  BBS  System  Tapes 

Summary 

Even  with  the  COBRAS  I/E  experience  and  materials,  development  of  the  COBRAS  El  BBSE 
was  a  complex  process.  The  initial  development  described  here  resulted  in  preparation  of  TSPs 
ready  for  trial  implementation.  The  next  section  (Section  4)  describes  the  BBSE  trial 
implementation  and  evaluation  plan,  the  findings,  and  the  results  of  that  trial,  including  the 
revisions  made  to  the  exercise  and  TSP 
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Section  4:  Trial  Implementation  of  the  Brigade  and  Battalion  Staff  Exercise 

The  key  evaluation  event  during  the  development  of  the  COBRAS  EH  BBSE  was  the  trial 
implementation  conducted  at  Fort  Lewis,  Washington,  with  3  Bde,  2  ID.  This  trial  involved 
personnel  representative  of  the  target  multiechelon  training  audience  using  the  TSP  as  designed 
in  a  full  implementation. 

This  section  describes  that  trial  and  the  trial  results.  Three  topics  are  discussed: 

•  Trial  Implementation:  This  part  describes  the  specifics  of  the  formative  evaluation  in 
terms  of  the  purpose  of  the  trial,  the  evaluation  methods  and  restrictions/constraints, 
and  the  evaluation  staffing  plan. 

•  Trial  Implementation  Results:  This  part  summarizes  feedback  provided  by 
participating  unit  members  and  observations  made  by  COBRAS  developers. 

•  Program  Revisions:  This  part  describes  the  extent  of  revisions  that  were  made  to  the 
TSP  as  a  result  of  the  trial. 


Trial  Implementation 

The  purpose  of  the  trial  implementation  was  to  evaluate  the  effect  of  training  program 
design  decisions  on  perceptions  of  training  benefits.  The  design  characteristics  of  interest  were: 
the  multiechelon  focus,  24-hour  operations,  deployed  CPs,  and  in-stream  observation  and 
feedback  approach. 

During  the  trial  implementation,  exercise  developers  observed  the  planning,  preparation, 
and  conduct  of  the  exercise  and  performed  various  data  collection  activities.  Although 
developers  were  prepared  to  assist  in  implementation  as  necessary  in  order  to  ensure  that  the 
exercise  could  be  conducted,  the  intent  was  to  operate  in  a  “hands-off’  mode  as  much  as 
possible. 

In  general,  the  exercise  was  implemented  according  to  the  guidance  provided  in  the  TSP. 
The  BCT  participated  with  one  battalion  being  observed;  the  other  two  battalions  were 
roleplayed  in  CPs  at  BBS  workstations.  The  CPs  for  the  brigade  and  the  observed  battalion  were 
emplaced  in  an  area  adjacent  to  the  simulations  center.  The  exercise  operated  on  a  full  24-hour 
schedule,  with  a  continuously  running  scenario  timeline. 

The  only  significant  ways  in  which  the  trial  did  not  fully  test  the  TSP  were: 

•  As  had  been  the  case  for  previous  trials,  the  observers  were  trained  professional 
observers  from  a  training  support  brigade.  Thus  their  experience  with  live  and 
constructive  simulation  based  training  was  extensive. 

•  Although  copying  and  assembly  of  the  TSP  materials  went  more  smoothly  than  ever 
before,  distribution  of  the  materials  was  somewhat  uneven.  Simulation  site  personnel 
received  the  BBS  materials  well  in  advance  of  the  exercise,  as  did  the  brigade 
commander  and  staff.  Observers  received  their  guides  and  materials  1-2  days  before 
the  exercise.  Few  of  the  members  of  the  observed  battalion  actually  received  their 
guides  at  all. 
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•  The  Forward  Support  Battalion  (FSB)  commander  participated  from  a  CP  located 
some  15  kilometers  from  the  exercise  area.  In  part,  this  was  because  the  FSB  had 
already  planned  to  conduct  their  own  training  during  the  trial  period,  and  wanted  to 
take  advantage  of  the  planned  opportunity  to  fully  set  up  and  use  their  CP. 

•  The  model  called  for  a  senior  officer,  preferably  from  the  brigade’s  higher 
headquarters,  to  serve  as  the  Exercise  Director.  However,  because  there  was  no  one 
of  that  level  available,  the  individual  who  was  soon  to  become  the  brigade’s  executive 
officer  (XO)  filled  the  role,  with  some  assistance  from  the  Senior  Observer,  the 
officer  in  charge  (OIC)  of  the  simulation  center,  and  the  COBRAS  team  members  on 
site.  This  actually  worked  very  well,  because  it  gave  both  the  commander  and  the 
future  XO  a  chance  to  begin  synchronization  of  their  abilities  and  expectations. 

•  Because  the  performance  objective  approach  was  new,  only  eight  complete 
performance  objectives  were  used  in  the  exercise.  This  was  somewhat  more  than 
should  be  included  in  a  single  implementation,  and  stretched  the  limits  of  the  abilities 
of  the  observer  team  and  the  BCT  to  focus  on  the  full  complement  of  objectives. 
Nonetheless,  it  meant  that  only  half  of  the  projected  performance  objectives  could  be 
tested. 

COBRAS  staff  involvement  in  controlling  the  implementation  was  less  extensive  than  it 
had  been  for  previous  trials,  in  part  because  two  of  those  trials  had  also  occurred  in  3  Bde,  2  ID. 
Only  one  pre-implementation  trip  was  conducted  to  coordinate  the  exercise.  The  visit  was 
supplemented  by  telephone  conversations  to  ensure  that  the  TSP  materials  were  being  correctly 
assembled. 

For  the  conduct  of  the  trial  implementation  itself,  14  of  the  COBRAS  staff  were  on  site 
before  the  interactor  and  roleplayer  training  began,  two  days  prior  to  the  exercise  start.  All 
requests  for  assistance  were  noted  for  later  incorporation  into  the  materials. 

Trial  implementation  data  were  collected  by  means  of  four  general  methods,  as  described 
in  Section  2: 

•  Structured  interviews  were  conducted  with  key  participants,  including  all  three 
members  of  the  exercise  administrators  (the  simulation  center  OIC,  COBRAS 
Coordinator,  and  Blue  Forces  Controller),  all  observers,  the  commanders  and  staffs  of 
the  brigade  and  the  observed  battalion,  the  individual  representing  the  Division 
Assistant  Chief  of  Staff,  Operations  (G3)  at  EXCON,  and  the  OPFOR  Controller. 

The  interview  protocols  had  been  prepared  before  the  trial  began  in  order  to  ensure 
that  attention  was  paid  to  critical  evaluation  issues. 

•  COBRAS  developers  observed  each  phase  of  the  implementation  process  and  noted 
problems  and  areas  for  improvement. 

•  Questionnaires  were  prepared  and  distributed  to  the  training  audience  members, 
observers,  roleplayers,  and  interactors.  The  questionnaires  contained  a  mixture  of  two 
types  of  items:  5-point  scale  response  items  (e.g.,  disagree-agree)  and  open  ended 
items  (e.g.,  what  additional  initialization  instructions  were  needed?). 
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•  Group  discussions  at  the  conclusion  of  the  exercise  were  used  to  obtain  more  in-depth 
information  and  perceptions  of  the  training  quality  and  value,  and  ideas  about  exercise 
features  that  should  be  strengthened,  modified,  or  dropped. 

Trial  Implementation  Results 

The  results  from  the  participant  interviews  and  questionnaires  at  the  trial  implementation 
of  the  COBRAS  HI  BBSE  are  described  below.  The  main  topic  of  interest  -  perceived  training 
benefit  -  is  discussed  first,  and  is  then  examined  specifically  in  terms  of  the  performance 
objectives  approach  and  the  observation  and  feedback  system. 

Perceived  Training  Benefit 

Perceptions  of  training  benefit  from  the  brigade  and  battalion  training  audience  members 
were  of  particular  interest.  There  were  several  questions  addressed  to  the  training  audience 
concerning  global  perceptions  of  training  benefit,  as  shown  in  Table  3.  The  exercise  was 
perceived  as  beneficial  for  individuals  and  for  the  brigade  and  battalion  overall.  Even  an 
experienced  brigade  or  battalion  staff,  they  felt,  would  find  the  exercise  effective. 

Table  3 

Training  Audience  Perceptions  of  Exercise  Utility 


Echelon 

Not  At  All  or  Moderate  Extent 

Slight  Extent 

Great  or  Very 
Great  Extent 

N 

To  what  extent  was  your  participation  worth  the  time  spent? 

Brigade 

0(0%)  3(16%) 

16  (84%) 

19 

Battalion 

1  (4%)  8  (33%) 

15  (63%) 

24 

To  what  extent  are  you  more  prepared  for  a  CTC-like  experience  than  before  the  exercise? 

Brigade 

2(11%)  1(5%) 

16  (84%) 

19 

Battalion 

1  (4%)  10  (42%) 

13  (54%) 

24 

To  what  extent  is  the  brigade  as  a  whole  more  prepared  for  a  CTC-like  experience  than  before  the 
exercise? 


Brigade  0(0%)  4(21%)  15(79%)  19 

Battalion  2(8%)  10(42%)  12(50%)  24 

To  what  extent  is  the  exercise  effective  for  experienced  brigade  or  task  force  staffs? 

Brigade  1(5%)  3(16%)  15(79%)  19 

Battalion  3(12%)  9(38%)  12(50%)  24 


Observers  were  also  asked  about  the  overall  benefits  of  the  exercise,  both  for  the  training 
audience  and  for  themselves  as  observers  (Table  4).  They  were  mostly  positive  about  the 
benefits  they  derived  from  participating  in  the  exercise.  Brigade-level  observers  were  positive 
about  the  value  to  the  training  audience;  battalion  observers  were  positive  but  not 
overwhelmingly  so.  Two  specific  comments  bear  repeating; 

•  “BBSE  has  great  potential.  But  all  personnel  need  to  be  proficient  as  individuals 
before  the  collective  part  begins.” 
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“I  think  that  BBSE  has  good  potential/promise  to  train  staffs  at  all  levels.” 


Table  4 

Observer  Perceptions  of  Exercise  Utility 


Echelon 

Not  At  All  or 
Slight  Extent 

Moderate  Extent  Great  or  Very  Great 

Extent 

N 

To  what  extent  did  you  benefit  from  participating  in  the  exercise  as  an  observer? 

Brigade  Observers 

2(18%) 

2(18%)  7(64%) 

11 

Battalion  Observers 

2  (22%) 

4  (44%)  3  (33%) 

9 

To  what  extent  do  you  feel  that  this  type  of  training  is  useful  for  the  training  audiencel 

Brigade  Observers 

0 

1(9%)  10(91%) 

11 

Battalion  Observers 

2  (22%) 

2  (22%)  5  (56%) 

9 

The  Exercise  Director  indicated  that  the  brigade  was  more  prepared  for  a  CTC  rotation 
than  they  had  been  before  the  exercise.  He  also  felt  that  his  participation  was  worth  the  time  and 
effort,  and  stated  that  his  review  of  OPFOR  doctrine,  brigade-level  decision-making,  and  battle 
staff  command  and  control  were  particularly  beneficial. 

Finally,  there  were  many  comments  from  the  training  audience  members  about  the 
positive  aspects  of  the  exercise.  These  are  presented  in  Table  5.  Respondents  saw  benefits  in  a 
wide  variety  of  areas,  from  practicing  staff  processes  in  planning  and  information  management  to 
understanding  of  their  assets  and  relationships  with  other  elements  of  the  division,  to 
opportunities  to  work  with  and  assess  sections  and  subordinates. 

Perceptions  of  the  Performance  Objectives  Approach 

Training  audience  members  and  observers  were  asked  about  the  unit’s  ability  to  perform 
each  performance  objective  before  the  exercise,  and  about  their  ability  to  perform  after  the 
exercise  was  completed  (both  questions  were  asked  after  the  exercise).  The  results  are  shown  in 
Table  6. 

The  number  of  respondents  commenting  on  each  objective  varied  across  the  performance 
objectives.  In  many  cases,  members  of  the  training  audience  did  not  feel  qualified  to  offer  a 
judgment  of  the  unit’s  proficiency  on  the  objective.  This  was  particularly  true  for  objectives  that 
seemed  to  focus  on  a  relatively  narrow  slice  of  the  training  audience  (e.g.,  performance  objective 
8  Integrate  CSS  Estimates  in  Decision-Making).  Similarly,  for  the  objectives  that  were  directed 
specifically  toward  the  brigade  staff  (performance  objectives  3  and  8)  or  toward  the  battalion 
staff  (performance  objectives  4  and  7),  members  at  the  other  echelon  were  reluctant  to  respond. 
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Table  5 

Training  Audience  Comments  on  Exercise  Benefits  (Number  of  Respondents  Making 
Comment) 


Staff  Planning  Processes 

Learned  a  great  deal  about  planning  process  (9) 
Learned  some  new  techniques  on  R&S  planning  and 
execution  (3) 

Doctrinal  procedures  for  parallel  planning  (3) 
Information  Management 
Streamlined  the  information  flow  within  the  TOC  (3) 
Learned  about  TOC  OPS  (7) 

Information  management  for  Bn  S3  shop 

Use  of  Assets 

Better  understanding  of  synching  Corps  assets 
Learned  more  about  DS  MI  company  type  assets  that 
usually  don’t  have  access  to 
Synchronization  of  critical  information  requirements, 
collection  and  decision  support 
Specific  Section  or  Task  Training 
How  important  ADA  EW  is,  educate  at  all  levels 
Trained  the  296  FSB  LNO 
I  have  better  trained  TF  FSNCOs 
My  enlisted  personnel  in  the  FSE  are  now  trained  to 
where  I’m  confident 
More  experience  in  job  position 
More  face  time  with  maneuver 
Continue  to  develop  better  ways  to  track  CSS  (2) 

RTO  procedures  (improvement) 

Tested  &  refined  Engineer  annex 
National  Training  Center  Preparation 
Understand  accelerated  NTC  battle  rhythm  (5) 
Familiarization  with  Bde  TACSOP,  refine  new 
TACSOP  (4) 

Got  to  work  with  the  new  Bde  staff  under  stress 
Opportunity  for  Assessment 
Good  outside  look 

Provided  an  excellent  assessment  opportunity 
Identified  weaknesses  and  strengths,  will  guide 
future  training  (5) 

General 

Better  understanding  of  doctrine _ 


Learned  extent  of  knowledge  on  mission  analysis 
procedures  (2) 

Battle  tracking  (2) 

Learning  brigade’s  order  process  rhythm  (6) 

Information  systems  glimpse  at  future  training 
The  Bde  TOC  learned  to  crosstalk  better 
Must  have  a  detailed  tracking  system  in  case 
TOC  is  destroyed 

Better  understanding  of  how  to  coordinate  with 
DIVARTY 

Working  knowledge  of  how  TF  integrates  its 
branches 

Increased  knowledge  in  synchronization  of  fires 

Troop  leading  procedures  at  company  level. 
Rehearsal  preparation 

Duty  positions/responsibilities  within  S3  shop 
S4  can’t  plan  and  run  a  CTCP  at  the  same  time; 

needs  help  from  S 1 . 

Must  use  forecasting  staff  guide 
Integrated  w/  FSE  &  CSS  at  Bde  level 
FASCAM  planning 

Build  teamwork  within  the  battle  staff,  increased 
staff  integration  (23) 

The  need  for  organization  to  make  maximum  use 
of  the  time  available 

Got  to  watch  my  soldiers  (new)  work  under 
stress  without  any  sleep 

Trained  in  a  good  no  distraction  environment 
A  chance  to  assess  the  strengths  and  weaknesses 
of  the  S2  shop 
Experience  in  my  job  (4) 

Net  higher  entry-level  skills  for  all  participants 
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Table  6 

Training  Audience  Perceptions  of  Performance  Objective  Proficiency 


Echelon 


Number  Reporting 
Increase  No  Change 


Decrease 


1.  Conduct  Clearance  of  Indirect  Fires  Procedures 

Brigade  7(41%)  9(53%) 

Battalion  12  (57%)  9  (43%) 

2.  Plan  and  Implement  Brigade  Air  Defense  Early  Warning 

Brigade  10  (56%)  8(44%) 

Battalion  11(55%)  9(45%) 

3.  Manage  Information  Within  the  Brigade  Command  Posts 

Brigade  17(90%)  2(10%) 

Battalion  7  (70%)  3  (30%) 

4.  Manage  Information  Within  the  Task  Force  Command  Posts 

Brigade  5  (100%)  0 

Battalion  18  (86%)  2  (9%) 

5.  Plan  and  Manage  Reconnaissance  Within  the  Brigade 

Brigade  14  (74%)  5  (26%) 

Battalion  17  (74%)  5  (22%) 

6.  Conduct  Parallel  Planning  Within  the  Brigade 

Brigade  13(68%)  6(32%) 

Battalion  17(71%)  6(25%) 

7.  Plan  and  Execute  Engagement  Areas 

Brigade  2  (50%)  2  (50%) 

Battalion  14  (64%)  7  (32%) 

8.  Integrate  Combat  Service  Support  Estimates  in  Decision-Making 

Brigade  11(69%)  5(31%) 

Battalion  5(71%)  2(29%) 


1  (6%) 
0 

0 

0 

0 

0 

0 

1  (5%) 
0 

1  (4%) 
0 

1  (4%) 
0 

1  (4%) 

0 

0 


N 


17 
21 

18 
20 

19 

10 

5 

21 

19 

23 

19 

24 

4 

22 

16 

7 


In  general,  respondents  indicated  that  their  unit  was  more  proficient  and  better  prepared 
on  the  performance  objectives  after  the  exercise  than  they  had  been  before  the  exercise. 
Comments  recorded  during  interviews  or  provided  on  the  questionnaires  also  indicated  that  the 
exercise  was  instrumental  in  improving  performance. 

The  Brigade  Commander  indicated  that  the  flexible  approach  of  tailoring  the  training 
emphasis  through  selection  of  performance  objectives  was  viable.  With  input  from  the  brigade 
XO  and  operations  and  training  officer  (S3),  he  felt  that  he  was  able  to  select  topics  and  focus 
training.  At  the  same  time,  the  entire  battle  staff  was  involved  and  receiving  training  even  if  not 
the  focus  of  an  observed  performance  objective.  He  stated  that  the  performance  objectives  allow 
commanders  to  “lay  the  groundwork  for  a  shared  vision  for  tactical  operations  and  commander’s 
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intent,”  and  that  the  approach  is  “especially  viable  for  fostering  a  command  environment  within 
the  brigade  and  developing  the  inter-staff  relationships  vital  for  operations.” 

The  Battalion  Commander  felt  that  the  parallel  planning  performance  objective  was 
sufficient  to  involve  the  battalion’s  training  of  decision-making.  The  focus  allowed  the  battalion 
to  validate  their  CP  and  tactical  standing  operating  procedures  (TACSOP),  and  contributed  to 
training  for  the  entire  battle  staff  in  all  areas  of  decision-making,  coordination  with  the  brigade 
staff,  and  information  flow.  However,  because  the  missions  were  not  linked,  the  staff  needed  to 
start  planning  from  the  beginning  each  time.  He  felt  that  this  approach  does  not  facilitate  the 
staff  being  able  to  use  their  shared  situational  awareness  in  planning  the  next  mission. 
Additionally,  significant  training  value  for  the  battalion  will  be  lost  if  the  battle  staff  is  deployed 
at  workstations  as  opposed  to  a  CP. 

The  Senior  Observer  was  veiy  positive  about  the  performance  objective  approach  as  a 
means  of  describing  exercise  behavior  and  performance.  He  compared  the  performance 
objectives  to  other  existing  doctrinal  performance  measures,  specifically  mentioning  ARTEP 
tasks,  and  stated  that  he  knew  of  nothing  “comparable  or  as  good  out  there.”  The  performance 
objectives  caused  both  the  observers  and  training  audience  to  look  at  issues  in  a  unified  way.  He 
liked  the  techniques  and  procedures  and  the  assessment  guide,  noting  that  the  instructions  were 
clear  and  easy  to  follow;  it  was  “really  great  to  have  everything  in  one  package.”  He  stated  that 
“the  whole  Army  needs  this  badly  and  quickly”  and  that  it  would  especially  benefit  the  National 
Guard:  “There  is  not  a  Guard  staff  out  there  that  would  not  benefit  from  the  [performance 
objective]  approach.” 

At  the  battalion  level,  the  battle  staff  and  observers  recommended  avoiding  performance 
objectives  that  require  the  battle  staff  to  be  “on  the  ground.”  Because  of  the  limitations  of  the 
simulation,  units  are  unable  to  fully  perform  those  activities.  For  example,  although  the  unit 
could  gain  some  value  through  planning  and  coordination  of  an  engagement  area,  they  were  not 
able  to  “construct”  one.  Other  comments  about  the  performance  objectives  included: 

•  The  recon  [performance  objective]  is  a  good  product  as  written.  Recommend  S3 
observer  also  has  input  and  tasks  identified  for  maneuver. 

•  Parallel  planning  [performance  objective]  good.  It  is  a  good  guide  for  observer  and 
unit  evaluated. 

•  The  information  management  [performance  objective]  could  be  more  specific  to 
include  some  of  the  information  in  the  reference  MTP.  Had  I  not  spent  time  in  TOCs, 
providing  a  good  evaluation  would  have  been  difficult  using  the  [performance 
objectives]. 

•  I  like  the  technique  of  using  [performance  objectives]  for  observing  a  unit,  rather  than 
the  traditional  method  of  “looking  at  everything.”  I  recommend  we  use  this  technique 
in  all  training  events  at  all  echelons. 

When  interviewed,  the  Exercise  Director  commented  that  the  performance  objectives 
“. . .  provide  a  great  resource  of  how  specific  battle  staff  functions  should  be  executed.  The 
information  found  in  the  [performance  objectives]  is  invaluable  to  both  the  O/Cs  and  to  the 
commander/staff.  Clearly,  other  [performance  objectives]  should  be  developed.” 
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Observation  and  Feedback  System 

According  to  the  Brigade  Commander,  the  feedback  system  was  appropriate.  It  allowed 
the  observer  to  focus  on  his  area  in  detail,  minimizing  the  “broad-based  general-type”  comments. 
It  resulted  in  minimal  interference  with  the  brigade  battle  staff  during  the  conduct  of  the  exercise. 

Small  group  informal  feedback  sessions  are  most  “beneficial”  as  they  place  soldiers  more 
at  ease  and  encourage  more  participation  by  noncommissioned  officers  and  enlisted  personnel. 
Generally  a  single  observer  for  a  performance  objective  is  sufficient.  However,  that  means  they 
may  not  possess  the  expertise  across  the  BOS  involved  with  the  objective.  The  observers  should 
provide  comments  from  their  observations  to  drive  the  battalion  discussion  during  the  feedback 
sessions. 

The  S3  of  the  observed  battalion  stated  that  the  system  allows  the  staff  to  revise  and 
refine  its  procedures  as  opposed  to  a  traditional  “checklist”  AAR.  Observers  should  refrain  from 
coaching  if  it  interferes  with  the  interaction  of  the  staff.  They  should  also  have  a  working 
knowledge  of  the  unit’s  standing  operating  procedures,  so  that  all  coaching  is  in  relation  to  those 
procedures.  Feedback  sessions  should  be  restricted  to  breaks  in  the  process  or  action. 

The  Brigade  Senior  Observer  noted  that  the  training  audience  generally  did  not  respond  to 
one  method  of  feedback  versus  another,  but  that  he  liked  the  flexibility  within  the  system  that 
enables  a  unit  to  conduct  its  own  feedback.  He  believed  that  multiple  feedback  sessions  are  a 
must,  encouraging  the  unit  to  conduct  their  own  discussions  using  the  assessment  guide.  He 
preferred  “informal”  feedback  sessions,  and  stated  that  the  assessment  guides  “did  an  excellent 
job  of  modeling  this  type  of  [feedback  session].”  He  also  indicated  that  observers  that  come 
from  a  “sister”  brigade  would  be  perfectly  capable  of  performing  the  duties  of  observers  with  this 
TSP;  a  dedicated  observer  team  was  not  required. 

The  observers  used  the  assessment  questions  provided  in  the  materials  as  the  basis  for  all 
the  feedback  sessions,  although  they  modified  some  to  meet  specific  situations.  They  believed 
them  to  be  complete  and  easy  to  use  in  preparing  for  a  feedback  session.  None  of  the  25  assigned 
observers  expressed  any  problem  understanding  the  performance  objective  approach.  The 
observers  felt  that  the  joint  introduction  to  the  system  given  by  the  COBRAS  staff  was  invaluable 
in  orienting  everyone  to  the  system.  They  recommended  that  something  similar  in  content  be 
developed  as  a  future  introduction. 

All  participants  in  the  trial  agreed  that  if  anything  should  be  dropped  from  the  system,  it 
was  the  “execution  AAR.”  This  session  was  not  a  part  of  the  TSP  design,  but  had  been  added  by 
the  observer  team  as  an  exercise  closure  mechanism.  It  focused  more  on  the  execution  itself, 
rather  than  on  the  performance  objectives. 

Program  Revisions 

The  exercise  design  features  -  24  hour  operations,  deployed  CPs,  and  multiechelon  and 
staff  section  focus  -  were  expected  to  have  noticeable  effects  on  the  perceptions  of  realism  and 
exercise  benefits.  To  gauge  the  effect,  we  looked  for  comments  about  the  exercise  realism  or 
benefits  attributable  to  those  features.  Surprisingly,  we  found  few  comments  directly  for  or 
against  any  of  the  features.  Comments  about  the  realism  of  the  exercise  tended  to  be  phrased  as 
suggestions  concerning  ways  to  improve  the  exercise  realism  (Table  7).  However,  as  noted 
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earlier,  several  participants  stated  that  they  learned  how  to  operate  better  within  their  sections 
because  of  the  CP  setup  and  the  24-hour  operations. 


Table  7 

Training  Audience  Suggestions  for  Improving  Exercise  Realism 


Roleplayers  and  scripted  messages 

More  detailed  information  for  providing  cues  and 
responses  from  Division  (4) 

Simulation 

Update  terrain  in  computer  so  that  what  is  go/  no-go 
on  map  and  in  reality  is  what  computer  allows. 

Division  early  warning  and  ADA  systems  [should  be] 
functioning  properly  &  doctrinally  (2). 

Combat  Service  Support 

Make  the  game  so  CO’s/lSGs  are  forced  to  report 
status  to  the  CTCP  like  they  would  at  a  CTC  rotation. 

Develop  the  concept  of  the  DISCOM  HQ. 

Better  CSS  interaction  to  involve  Bn  staff  (2). 

Scenario 

Make  weather  more  significant  in  at  least  one  battle, 
i.e.,  a  5-7000’  cloud  deck. 

Develop  more  variety  of  flex  missions. 

Use  continuous  storyline  where  the  impact 
(success/loss)  of  one  carries  over  into  the  next  (9) 

Tactical  presentation,  situation,  and  materials 

Update  TOE  in  game  w/  current  force  structure  (3). 
More  NBC. 

Use  more  combat  multipliers. 

Deployed  Command  Posts,  Training  Audience 

Better  commo  assets  (MERT,  TACFAX)  (3) 

Make  it  longer.  Force  them  to  live  like  they  are  at  a 
CTC  (2). 

Move  the  BSA  closer  to  [simulation  center],  I  spent 
too  much  time  on  the  road  collecting  reports. _ 


Need  experienced  people  at  EXCON  to  provide 
realistic  Division  responses. 

Less  reliance  on  computerisms  (e.g.,  straight  line 
distances  for  maneuver)  in  shaping/  conducting 
battle  (2). 

The  BBS  log  algorithm  uses  too  much  fuel  (2). 

Allow  use  of  my  log  reports.  They  are  much 
more  detailed  than  the  generic  Division  log 
reports  (2). 

Make  the  exercise  more  realistic  for  CSS  (4). 

Take  out  full  day  lull  in  fighting  between  defense 
&  offense. 

Develop  the  OPORDs  for  other  terrain. 

Choose  terrain  for  defense  that  would  support 
Bde  EA. 

Don’t  put  things  in  the  OB  if  you  never  intend  to 
use  it  all,  or  if  you  don’t  have  the  capability  to 
use  it  correctly  via  BBS. 

Make  the  staffs  play  “full  up”  (3). 

More  branch  reps  at  the  cell  as  operators 

Place  Bde/Bn  TOCs  more  than  walking  distance 
away  from  each  other.  It  will  enforce  LNOs  & 
radio  TOC  procedures  better  (3). _ 


All  in  all,  the  evaluation  results  from  the  trial  implementation  were  encouraging.  The 
BBSE  design  decisions  were  supported  (or  at  least  not  contraindicated),  and  the  BBSE  was  seen 
as  providing  training  benefit,  both  for  newly-formed  brigade  staffs  and  for  more  experienced 
staffs. 

Developers  both  observed  use  of  the  TSP  and  asked  questions  about  the  TSP  structure. 
These  were  aimed  primarily  at  the  components  that  were  not  already  tested  in  earlier  programs  - 
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the  performance  objectives  and  the  observation  and  feedback  system.  The  TSP  was  perceived  as 
adequate  in  content  and  structure.  Various  minor  errors  were  caught  (or  pointed  out  by 
participants),  all  of  which  were  noted  and  corrected.  But  no  serious  design  or  content  flaws  were 
discovered. 

The  performance  objective  set  used  in  the  trial  implementation  was  expanded  after  the 
trial.  Of  the  original  eight,  only  one  (Develop  Engagement  Areas)  was  dropped.  Participants  and 
developers  alike  found  that  the  most  important  aspects  of  that  objective  required  participants  to 
be  “on  the  ground,”  a  condition  that  could  not  be  replicated  with  the  BBSE.  Additionally,  eight 
new  performance  objectives  were  developed  to  provide  the  commander  with  a  wider  variety  of 
options.  The  final  set  of  15  performance  objectives  is  listed  in  Figure  2. 


Parallel  Planning  within  the  Brigade 
Conduct  Clearance  of  Indirect  Fires  Procedures 
Plan  and  Manage  Reconnaissance  within  the  Brigade 
Integrate  Logistics  Estimates  in  Decision-Making 
Manage  Information  within  the  Brigade  Command  Posts 
Plan  and  Implement  Brigade  Air  Defense  Early  Warning 
Develop  and  Execute  the  Brigade  and  Battalion  Plan  for  Fires 
Conduct  a  Combat  Health  Support  Rehearsal 
Decision-making  in  a  Time  Constrained  Environment 
Plan  and  Execute  a  Decision  Point 

Develop  and  Execute  the  Brigade  Concept  of  Mobility/Survivability 
Plan  for  and  Commit  a  Company-Size  Reserve 
Plan,  Integrate,  and  Manage  Smoke  Assets 
Manage  Information  Within  the  Task  Force  Command  Posts 
Military  Decision-Making  Process  in  the  Battalion 

Figure  2.  Final  set  of  performance  objectives  for  the  Brigade  and  Battalion  Staff  Exercise. 

There  were  many  indicators  for  future  development,  however.  While  the  BBSE  was 
observed  to  be  a  useful  training  tool  for  brigade  staffs  (especially  more  mature  staffs  preparing 
for  a  CTC  rotation  or  deployment),  other  areas  of  concern  in  brigade  operations  were  discussed. 
Some  of  these  areas  are  discussed  in  Section  6. 
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Summary 

The  trial  implementation  of  the  BBSE  was  part  of  an  ongoing  evaluation  effort  that 
required  considerable  time  and  effort.  However,  it  was  necessary  in  order  to  obtain  the 
information  from  both  users  and  observers  concerning  the  performance  objectives  approach,  the 
observation  and  feedback  system,  the  TSP  materials  relating  to  those  components,  and  the 
perceptions  of  overall  exercise  benefits. 
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Section  5:  Lessons  Learned  for  the  Training  Developers 

With  every  training  development  program,  we  add  to  the  lessons  learned  base  of 
information.  Over  the  past  9  years,  ARI  and  its  contractor  teams  have  accomplished  a  number  of 
R&D  efforts  to  incorporate  structure  into  battalion-  and  brigade-level  training  programs.  The 
preliminary  work  in  development  of  structured  training  scenarios  was  focused  on  the  definition  .. 
of  threat  conditions  (C.  H.  Campbell  &  Hoffman,  1990;  R.  C.  Campbell  &  Campbell,  1990; 
Doyle,  1990).  This  work  was  vastly  expanded  in  the  initial  development  and  expansions  to  the 
VTP  (Graves  &  Myers,  1997;  Hoffman  et  al.,  1995;  Hoffman,  1997;  Koger  et  al.,  1996).  Similar 
work  was  performed  for  exercises  using  the  Close  Combat  Tactical  Trainer  (CCTT)  for  platoon, 
company,  and  battalion  training  (Deatz  et  al.,  1998;  Flynn,  Campbell,  Myers,  &  Burnside,  1998). 
Both  COBRAS  I  (Graves  et  al.,  1997)  and  COBRAS  II  (C.  H.  Campbell  et  al,  1998),  focusing  on 
brigade  staff  training,  used  the  same  general  approach.  Initial  evaluations  of  such  programs  have 
indicated  that,  with  a  well-developed  training  structure,  simulation  can  contribute  substantially  to 
the  value  of  training  (Shlechter,  Bessemer,  Nesselroade,  &  Anthony,  1995).  Therefore,  the 
lessons  learned  do  not  address  instructional  features  any  further. 

Program  developers  and  sponsors,  however,  are  well  aware  that  the  structured  approach 
and  development  methodologies  can  always  be  improved.  To  that  end,  each  of  the  ARI  projects 
has  produced  a  report,  documenting  “lessons  learned”  that  may  be  useful  in  future  development 
efforts.  The  lessons  learned  during  the  COBRAS  HI  project  are  described  below. 

The  lessons  were  drawn  from  experience  throughout  the  project  period.  They  focus  on: 

•  needs  analysis  and  training  development, 

•  data  collection  in  trial  implementations,  and 

•  the  nature  of  lessons  learned. 

These  lessons  should  be  of  use  to  program  developers,  whether  they  are  design  scientists 
who  are  responsible  for  delineating  project  objectives  and  design  alternatives,  or  military  and 
instructional  specialists  who  construct  TSPs. 

Needs  Analysis  and  Training  Development 

Because  this  program  was  the  third  in  a  series  of  development  efforts,  and  because  three 
of  the  trial  implementations  were  conducted  with  the  same  unit,  the  BBSE  was  based  on  some 
clearly  identified  training  needs.  In  the  May  1997  trial,  the  Brigade  Commander  was  already 
attempting  to  modify  the  COBRAS  II  BSE  in  preparation  for  an  NTC  rotation.  His  highest 
priority  changes  included:  24-hour  operations,  deployed  CPs,  a  robust  and  challenging  OPFOR, 
multiechelon  focus,  and  concurrent  planning  and  execution.  The  decision  in  COBRAS  HI  was  to 
design  precisely  that  kind  of  exercise,  and  to  make  it  rigorous  enough  for  a  unit  that  is 
approaching  its  CTC  rotation.  As  a  result,  the  BBSE  was  welcomed  and  was  used  with  few 
modifications. 

LESSON:  Design  a  program  for  which  there  is  an  identified  need,  and  it 
will  be  used. 
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Data  Collection  in  Trial  Implementations 

This  lesson  was  one  that  was  learned,  yet  again,  the  hard  way.  The  questionnaires  and 
structured  interviews  addressed  the  performance  objectives  approach  and  the  observation  and 
feedback  system,  but  did  not  ask  detailed  questions  about  the  design  features.  The  questions 
were  asked  during  interviews  and  group  discussions,  but  only  because  the  developers  happened 
to  notice  the  omission. 

At  the  same  time,  the  questionnaires  asked  a  number  of  detailed  questions  about  the  TSP 
structure  and  utility.  The  TSP  structure  has  been  essentially  unchanged  over  the  past  four  years, 
so  there  is  little  information  gained  by  asking  about  preparation  activities  or  reading  time.  The 
information  needed  concerns  only  new  features  within  the  TSP  (e.g.,  the  performance  objectives 
materials). 

LESSON :  Data  collection  in  trial  implementations  should  address  all  of 
the  essential  questions  and  none  of  the  non-essential  questions.  The 
proposed  use  for  the  data  should  be  specified  before  the  data  are  collected. 

The  Nature  of  Lessons  Learned 

As  the  number  of  “lessons  learned”  continues  to  grow,  we  are  continually  reminded  to 
learn  old  lessons  better,  even  as  we  amass  new  ones.  There  is  a  need  to  review  and  analyze  those 
lessons  in  the  course  of  development  projects.  While  all  of  the  information  is  available  in  the 
separate  reports,  it  would  be  a  benefit  to  developers  to  have  a  single  compendium  or  database 
containing  lessons  and  even  the  larger  lessons  that  emerge  from  content  analysis  of  the  individual 
lessons. 

LESSON:  Developers  need  to  review  and  heed  lessons  learned  in  previous 
work. 


Summary 

This  section  has  presented  and  discussed  some  of  the  lessons  learned  from  the  COBRAS 
in  development  and  trial  implementation  effort.  Lessons  were  noted  during  all  phases  of  the 
project,  from  the  initial  design  through  implementations. 

The  final  section  in  this  report  describes  some  issues  that  remain  to  be  addressed.  Some 
are  underway  already.  Others  will  be  likely  to  demand  attention  in  the  near  (5-year)  future,  if  the 
U.S.  Army  is  to  continue  with  steady  progress  toward  its  Force  XXI  goals  in  training  and 
readiness. 
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Section  6:  Future  Directions 


Even  as  the  COBRAS  HI  development  and  refinement  was  being  completed,  additional 
training  R&D  programs  had  begun.  They  are  designed  to  address  training  needs  for  brigade  and 
below  that  were  been  pointed  out  by  users,  observed  by  the  COBRAS  developers,  or  suggested 
by  the  Force  XXI  Training  Program  personnel  and  other  stakeholders. 

The  list  of  training  needs  presented  in  the  COBRAS  II  report  included  two  needs 
addressed  in  COBRAS  HI:  an  expanded  multiechelon  version  of  the  BSE,  still  in  BBS,  with  24- 
hour  operations,  deployed  CPs,  more  intense  battle  rhythms,  and  more  robust  and  aggressive 
OPFOR  activity;  and  a  multiechelon  BSE  incorporating  linked  virtual  (Simulation  Networking 
[SIMNET]  and  reconfigurable  simulations)  and  constructive  (BBS  and  Modular  Semi- 
Automated  Forces  [ModSAF])  simulation  environments.  Another  identified  need  is  being 
addressed  in  ARI’s  project  entitled  Implementation  and  Support  for  the  Assessment  of  Force  XXI 
Training  Program  Products  (ISAT).  Exercises  for  use  by  platoons,  companies,  and  battalions  in 
CCTT  were  developed  in  two  ARI  programs  entitled  Structured  Training  for  Units  in  the  Close 
Combat  Tactical  Trainer(STRUCCTT)  (I  and  II).  The  ARI’s  Commander’s  Integrated  Training 
Tool  ( CUT)  development  project  is  focused  on  design  and  prototype  development  of  an 
automated  tool  to  help  commanders  and  unit  trainers  access  TSPs,  modify  existing  scenarios  and 
TSPs,  or  create  their  own  scenarios  and  TSPs. 

The  most  pressing  needs  that  have  been  identified  for  development,  and  that  are  still  not 
being  addressed,  are  listed  below.  The  list  is  shorter  than  it  was  a  year  ago,  but  is  still 
formidable,  and  only  a  few  of  these  identified  needs  are  likely  to  be  addressed  anytime  soon. 

•  Additional  BSEs  and  BBSEs  on  Korean  and  European  terrains 

•  Parallel  versions  of  the  BSE  and  BBSE  for  light-heavy  units 

•  A  maneuver  battalion  version  of  the  BSE,  incorporating  CSS  emphasis  and  the  full 
planning  and  preparation  phases 

•  Additional  vignettes  focused  on  maneuver  battalion  training  needs 

•  Digital  environment  parallels  for  the  programs  already  developed  and  for  those  listed 
above. 


The  Next  Step:  Force  XXI  Training  Program  -  Digital 

The  COBRAS  in  work  represented  an  expansion  to  the  COBRAS  II  BSE  in  several  areas, 
including  the  training  audience,  implementation  model,  and  simulation  environment.  COBRAS 
HI  comprised  two  major  components:  a  constructive  simulation  Brigade  and  Battalion  Staff 
Exercise  and  a  linked  constructive  and  virtual  simulation  exercise  known  as  the  Synthetic 
Theater  of  War  Exercise. 

In  one  current  effort,  developers  are  squarely  in  the  digital  environment.  The  project  is 
entitled  Force  XXI  Training  Program  -  Digital  (FXXITP-D),  and  it  will  require  developers  to 
analyze  performance  in  the  digital  environment  of  Force  XXI  Battle  Command  Brigade  and 
Below  (FBCB2)  and  Army  Tactical  Command  and  Control  System  (ATCCS).  The  analysis  will 
yield  not  only  a  description  of  “digital”  performance  for  brigade  and  battalion  staffs,  but  will 
form  the  basis  of  a  plan  for  converting  the  Battle  Staff  Training  System  (BSTS),  vignettes,  BSE, 
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and  BBSE  to  digital-appropriate  training.  The  plan  will  be  evaluated  by  means  of  a  small-scale 
conversion:  a  single  set  of  computer-based  individual  training  modules  and  two  vignettes  for 
digitized  units  will  be  developed. 

The  Future  of  Training  Research 

The  purpose  of  this  report  was  to  describe  the  development  of  the  COBRAS  HI 
multiechelon  brigade  and  battalion  training  exercises  and  to  report  on  lessons  learned  for  training 
developers.  The  COBRAS  in  project  resulted  in  design  and  development  of  the  most  complex  of 
the  structured  simulation-based  training  programs  developed  by  the  ARI  Armored  Forces 
Research  Unit  at  Fort  Knox. 

As  detailed  in  Section  5,  there  have  been  numerous  developments  of  structured 
simulation-based  training.  The  various  TSPs,  along  with  the  formative  evaluation  results  and 
lessons  learned  detailed  in  the  final  reports,  serve  as  both  guides  and  models  for  future 
development.  Three  method  guides  for  development  have  also  been  published  by  ARI 
(C.H.  Campbell,  Deter,  &  Quinkert,  1997;  C.H.  Campbell,  Ford,  Campbell,  &  Quinkert,  1998; 
C.H.  Campbell,  Campbell,  Sanders,  Flynn,  &  Myers,  1995).  While  more  development  is  needed 
to  address  additional  training  needs,  the  research  on  how  the  training  should  be  developed  and 
what  the  TSPs  should  include  should  not  be  a  high  priority.  For  most  applications,  the  work 
should  be  transitioned  over  to  emphasize  development,  rather  than  research. 

Instead,  the  research  community  should  be  focusing  on  ways  to  make  the  training  more 
adaptable  and  more  accessible.  Each  user  finds  ways  to  “improve”  the  exercise  for  his  own 
situation,  and  tools  must  be  found  that  will  help  them  to  make  the  modifications  without 
destroying  the  intricate  structure  that  makes  the  training  so  valuable.  Additionally,  ways  must  be 
devised  for  conveying  the  TSPs  to  the  users.  At  the  time  of  this  writing,  the  COBRAS  II  and  III 
TSPs  for  the  BSE,  BBSE,  and  vignettes  are  still  not  available  to  most  units. 

The  research  community  should  also  be  addressing  different  modes  for  training  delivery. 
Current  approaches  that  use  paper-based  TSPs  might  be  better  served  by  CD-ROM  based  or  even 
video-assisted  guidance.  Embedded  training  methods  may  be  more  available  now  that  computers 
themselves  are  embedded  in  most  soldiering  systems. 

Because  summative  evaluation  of  training  effectiveness  at  the  brigade  and  battalion  levels 
is  very  difficult,  we  often  find  ourselves  assessing  acceptability  and  drawing  conclusions  about 
skill  acquisition  and  transfer  of  skills  to  field  settings.  One  ARI  project  (ISAT)  is  looking  more 
closely  at  the  issue  of  effectiveness.  Another  ARI  study  will  begin  shortly  to  examine  the 
coordination  efforts  that  could  facilitate  training  development  between  TRADOC  proponent 
schools  and  user  units.  More  such  research  efforts  are  needed. 

At  no  point  will  we  be  able  to  say  that  we  have  learned  everything  about  training  that  can 
be  learned.  Even  the  most  routine  and  mundane  training  development  efforts  and  training 
implementations  will  yield  valuable  lessons  learned.  But  the  research  efforts  need  to  be 
forcefully  and  purposefully  leading  the  way  in  discovering  ways  to  improve  training  design, 
delivery,  and  evaluation. 
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Appendix  A.  Acronym  and  Abbreviation  List 


1SG 

AA 

AAR 

AD 

ADA 

ADCOORD 

ARI 

ARTEP 

ATCCS 

AVN 

BBS 

BBSE 

BCT 

Bde 

BFA 

Bn 

BOS 

BSE 

BSTS 

CALL 

CCTT 

Cdr 

CHEMO 

CUT 

CLS 

COA 

COBRAS 

COL 

CP 

CS 

CSS 

CTC 

CTCP 

DA 

DATK 

DDMP 

DISCOM 

DrVARTY 


first  sergeant 

assembly  area 
after  action  review 
area  defense 
air  defense  artillery 
air  defense  coordinator 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Army  Training  and  Evaluation  Program 
Army  Tactical  Command  and  Control  System 
aviation 

Brigade/Battalion  Battle  Simulation 
Brigade  and  Battalion  Staff  Exercise 
Brigade  Combat  Team 
brigade 

battlefield  functional  area 
battalion 

battlefield  operating  system 
Brigade  Staff  Exercise 
Battle  Staff  Training  System 

Center  for  Army  Lessons  Learned 
Close  Combat  Tactical  Trainer 
commander 
chemical  officer 

Commander’s  Integrated  Training  Tool 
contracted  logistics  support 
course  of  action 

Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved 
Through  Simulation 
colonel 

command  post 
combat  support 
combat  service  support 
Combat  Training  Center 
combat  trains  command  post 

Department  of  the  Army 
deliberate  attack 

deliberate  decision-making  process 
Division  Support  Command 
Division  Artillery 


A-l 


DS 

direct  support 

EA 

engagement  area 

ENG 

engineer 

EXCON 

exercise  control 

EW 

electronic  warfare 

FA 

field  artillery 

FBCB2 

Force  XXI  Battle  Command  Brigade  and  Below 

FDC 

fire  direction  center 

FM 

field  manual 

FRAGO 

fragmentary  order 

FSB 

forward  support  battalion 

FSCOORD 

fire  support  coordinator 

FSO 

fire  support  officer 

FTX 

field  training  exercise 

FXXITP-D 

Force  XXI  Training  Program  -  Digital 

G1 

assistant  chief  of  staff,  personnel 

G2 

assistant  chief  of  staff,  intelligence 

G3 

assistant  chief  of  staff,  operations 

G4 

assistant  chief  of  staff,  logistics 

HICON 

higher  control 

HQ 

headquarters 

ID 

infantry  division 

ISAT 

Implementation  and  Support  Team  for  the  Assessment  of  the 
Force  XXI  Training  Program 

ldr 

leader 

LNO 

liaison  officer 

MDMP  modified  decision-making  process 

METT-TC  mission,  enemy,  terrain,  troops,  time,  and  civilian  considerations 

MI  military  intelligence 

ModSAF  Modular  Semi-Automated  Forces 

MP  military  police 

MTC  movement  to  contact 

MTOE  modified  table  of  organization  and  equipment 

MTP  Mission  Training  Plan 

MWSTC  Mounted  Warfare  Simulations  Training  Center 


NBC 

NCO 


nuclear,  biological  and  chemical 
noncommissioned  officer 


NTC 

National  Training  Center 

OB 

order  of  battle 

OIC 

officer  in  charge 

O/C 

observer/controller 

OPFOR 

opposing  forces 

OPORD 

operations  order 

OPS 

operations 

PAM 

pamphlet 

pit 

platoon 

R&D 

research  and  development 

R&S 

reconnaissance  and  surveillance 

RTO 

radio/telephone  operator 

SI 

personnel  officer  (adjutant) 

S2 

intelligence  officer 

S3 

operations  and  training  officer 

S4 

logistics  officer 

SIGO 

signal  officer 

SIMNET 

Simulation  Networking 

SIMUTA 

Simulation-Based  Multiechelon  Training  for  Armor  Units 

SOP 

standing  operating  procedures 

SOW 

statement  of  work 

SPA 

Staff  Performance  Analysis 

STARTEX 

start  of  exercise 

STOW 

Synthetic  Theater  of  War 

STRUCCTT 

Structured  Training  for  Units  in  Close  Combat  Tactical  Trainer 

TAC 

tactical  command  post 

TACFAX 

tactical  facsimile 

TACSOP 

tactical  standing  operating  procedure 

TF 

task  force 

TOC 

tactical  operations  center 

TOE 

table  of  organization  and  equipment 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

TSP 

training  support  package 

USAARMC 

U.S.  Army  Armor  Center 

VTP 

Virtual  Training  Program 

XO 

executive  officer 

< 
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Appendix  B.  Foundations:  Overview  of  the  COBRAS  I  AND  II  Programs 

The  original  Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved 
Through  Simulation  (COBRAS)  projects  (COBRAS  I  and  COBRAS  II)  resulted  in  development 
of  the  Brigade  Staff  Exercise  (BSE)  and  the  Battalion  Virtual  Training  Program  (VTP)  COBRAS 
Expansion.  These  exercises  were  developed  at  Fort  Knox,  Kentucky  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences’  (ARI)  Armored  Forces  Research  Unit,  with  the 
assistance  of  a  consortium  of  contractors.  The  COBRAS  I/II  exercises  provided  the  models  from 
which  the  COBRAS  ID  Brigade  and  Battalion  Staff  Exercise  was  to  be  designed. 

This  appendix  provides  a  concise  description  of  the  BSE  and  the  Battalion  VTP 
Expansion  and  some  discussion  of  key  development  processes.  The  intent  is  to  provide  the 
background  necessary  for  understanding  the  COBRAS  HI  project’s  development  processes  and 
outcomes,  given  that  most  of  these  processes  and  outcomes  were  based  on  activities  of  the 
COBRAS  I/B  projects.  The  two  major  topics  of  the  section  are: 

•  The  COBRAS  Brigade  Staff  Exercise:  This  section  provides  a  description  of  the  BSE 
and  key  development  processes,  including  the  development  of  training  objectives. 

•  The  Battalion  VTP  COBRAS  Expansion:  This  section  provides  a  description  of  the 
expanded  VTP  exercise  that  incorporated  combat  support  (CS)  and  combat  service 
support  (CSS)  activities,  and  presents  the  key  development  processes. 

The  COBRAS  Brigade  Staff  Exercise 

The  BSE  is  a  multi-mission,  large-scope  practice  exercise  that  focuses  on  the  interactions 
among  the  brigade  commander  and  his  staff  as  they  conduct  planning  and  employ  brigade  assets. 
This  focus  was  selected  due  to  indications  that  the  brigade  commander  and  his  staff  need 
structured  practice  opportunities  to  achieve  proficiency  in  basic  brigade  operations  of  planning 
and  synchronizing  assets.  The  program,  as  designed,  gives  the  commander  and  his  staff  a  chance 
to  practice  the  tasks  they  should  perform  as  they  fight  the  brigade  in  the  particular  battles  of  a 
structured  scenario.  Within  a  simulated  (Brigade/Battalion  Battle  Simulation  [BBS])  combat 
situation,  they  must  determine  what  has  to  be  done  on  the  battlefield,  who  does  it,  and  how  their 
actions  are  linked  to  actions  of  other  units  and  battlefield  operating  systems. 

This  description  of  the  BSE  describes  four  key  aspects  of  the  exercise: 

•  the  training  audience, 

•  the  training  objectives  and  their  development, 

•  the  exercise  implementation  model,  and 

•  the  structure  and  composition  of  the  training  support  package  (TSP)  that  includes  all 
the  information  required  to  implement  the  exercise. 
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Training  Audience 

The  core  training  audience  members  included  the  brigade  commander  and  his  primary 
staff  (i.e.,  Executive  Officer  [XO],  Adjutant  [SI],  Intelligence  Officer  [S2],  Operations  and 
Training  Officer  [S3],  Supply  Officer  [S4],  and  Fire  Support  Officer  [FSO]).  During 
COBRAS  I,  training  audience  members  also  included  the  special  staff  who  serve  as  links 
between  the  brigade  and  four  of  its  systems  (i.e.,  fire  support,  air  defense,  engineering,  and 
logistics).  These  linking  personnel  were  determined  to  be  the  following: 

•  the  air  defense  coordinator  (ADCOORD,  the  air  defense  artillery  battery  commander), 

•  the  engineer  (ENG,  the  engineer  battalion  commander), 

•  the  forward  support  battalion  commander  (FSB  commander),  and 

•  the  fire  support  coordinator  (FSCOORD,  the  direct  support  (DS)  artillery  battalion 
commander). 

For  COBRAS  n,  the  training  audience  was  expanded  to  include  five  more  brigade 
responsibilities: 

•  the  signal  officer, 

•  the  assistant  S3  chemical  officer, 

•  the  military  police  platoon  leader, 

•  the  Army  aviation  brigade  liaison  officer,  and 

•  the  DS  military  intelligence  company  commander. 

This  group  of  16  constitutes  the  primary  training  audience  for  the  BSE.  The  primary 
training  audience  was  operationally  defined  as  those  participants  for  whom  training  objective 
task  lists  would  be  generated,  observers  would  be  assigned,  and  after  action  review  (AAR) 
sessions  would  be  provided. 

Training  Objectives  and  Tasks 

One  of  the  most  definitive  features  of  the  COBRAS  BSE  is  its  set  of  exercise  training 
objectives  and  tasks.  With  a  focus  on  the  planning  and  synchronization  of  brigade  assets,  as  well 
as  a  special  emphasis  on  CSS  functions,  the  BSE  performance  objectives  cover  a  wide  range  of 
staff  activities.  These  activities  are  summarized  in  four  general  staff  performance  objectives: 

1.  Performance  of  the  full  mission  requirements  of  planning,  preparation,  and  execution 
(including  consolidation,  reorganization,  and  planning  for  follow-on  missions) 

2.  Performance  of  both  the  deliberate  decision-making  process  (DDMP6),  performed 


6  During  the  period  of  time  in  which  COBRAS  I  development  proceeded,  the  decision-making  process  was  titled 
“Deliberate  Decision-Making  Process”  (DDMP).  Later,  during  the  COBRAS  II  project,  the  name  was  officially 
changed  to  the  “Military  Decision-Making  Process”. 
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without  time  pressure,  and  a  modified  decision-making  process  (MDMP7),  performed 
under  time  constrained  conditions 

3.  Complete  production  of  planning  and  preparation  products,  including  interim 
products  and  inputs. 

4.  Integration  of  selected  CS  and  CSS  functions  into  the  staff  processes  of  planning, 
preparation,  and  execution. 

These  objectives  are  supported  by  arrays  of  brigade  staff  tasks  that  are  specified  for  each 
of  the  16  members  of  the  target  training  audience  for  each  of  the  three  missions  -  a  total  of  48 
task  lists.  The  tasks  are  consistent  with  current  doctrine,  as  defined  by  Army  manuals  such  as 
Army  Training  and  Evaluation  Program-Mission  Training  Plan  (ARTEP-MTP)  and  Field  Manual 
publications,  but  are  not  constrained  to  the  contents  of  these  documents.  Rather,  the  tasks 
include  living,  innovative,  and  adaptive  tasks,  which  are  descriptions  of  behaviors  that  underlie 
successful  and  exemplary  performance.  During  the  project,  the  cumulative  domain  of  these 
behaviors  was  termed  “undocumented  tasks”  to  differentiate  them  from  the  mainstream, 
primarily  ARTEP-based,  documented  tasks. 

The  methodology  used  to  identify  the  COBRAS  tasks  was  entitled  the  Staff  Performance 
Analysis,  or  SPA.  The  methodology  employed  roleplay  enactments  of  the  BSE’s  missions  and 
subsequent  probing  activities  to  determine  the  activities  of  the  commander  and  staff  during  the 
BSE’s  particular  missions.  Tasks  identified  during  the  SPA  met  certain  criteria  based  on  project 
objectives.  These  criteria  included  the  following: 

•  Content  relevance:  Performance  was  defined  by  the  mission,  enemy,  terrain,  troops, 
time,  and  civilian  considerations  (METT-TC)  of  three  specific  missions.  Unlike 
existing  generalizable  performance  descriptions,  the  SPA  and  its  outcomes  (i.e.,  tasks) 
were  to  be  specific  to  COBRAS  METT-TC. 

•  Functional  emphasis:  Developers  documented  the  processes  during  each  mission 
phase  (planning,  preparing,  executing  the  mission,  and  conducting  consolidation  and 
reorganization)  in  specific  functional  areas:  command  and  control,  CSS,  fire  support, 
air  defense,  mobility/countermobility/survivability,  intelligence,  and  maneuver.  The 
cells  in  the  matrix  formed  by  the  overlay  of  these  two  domains  (mission  phase  X 
functional  area)  were  to  serve  as  a  structure  for  identifying  tasks. 

•  Interactive  criticality:  Special  emphasis  was  placed  on  identifying  activities  which 
require  interactions  between  pairs  or  groups  of  training  audience  members. 

•  Observability:  Tasks  were  to  have  observable  processes,  products,  or  outcomes. 
Because  the  training  process  emphasized  the  contributions  of  observers  who  would 
coach  and  provide  feedback,  it  was  necessary  that  observable  task  indicators  be 
identified.  Important  as  they  are,  performance  requirements  such  as  reasoning. 


7  The  process  called  “Modified  Decision-Making”  in  the  COBRAS  I  work  was  referred  to  variously  as  the 
abbreviated  or  accelerated  decision-making  process  in  Field  Manual  (FM)  71-3  (Department  of  the  Army  [DA], 
1995)  and  the  combat  decision-making  process  in  FM  101-5  (DA,  1997). 
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thought  processes,  and  knowledge  cannot  be  readily  coached  and  evaluated  during 
performance  unless  there  are  observable  outcomes. 

•  Brigade  staff  process  focus:  The  focus  of  performance  for  the  training  participants  is 
only  on  their  roles  within  the  conduct  of  staff  processes.  This  is  especially  significant 
in  the  situations  of  those  individuals  who  have  command  responsibilities  in  addition 
to  brigade  staff  responsibilities.  For  example,  the  brigade  ENG  is  also  the 
commander  of  the  engineer  battalion,  but  tasks  involving  the  command  of  his 
battalion  were  not  to  be  addressed. 

The  Exercise  Implementation  Model 

The  implementation  model  can  be  defined  in  terms  of  four  exercise  aspects:  participants, 
simulation  site  layout,  preparation  for  the  exercise,  and  conduct  of  the  exercise.  The  discussion 
below  describes  these  aspects  to  depict  how  the  BSE  is  implemented. 

Exercise  participants.  The  BSE,  as  designed,  requires  a  total  of  1 1 1  participants,  in 
addition  to  the  16  included  in  the  primary  training  audience.  These  participants  fill  roles  ranging 
from  high-level  exercise  management,  to  the  roleplaying  of  subordinate  unit  staffs,  to  operating 
BBS.  Additional  participants  could  be  inserted  into  any  of  the  role  types,  but  decreasing  the 
exercise  staff  would  reduce  the  training  benefit  of  the  brigade  staff.  The  types  and  positions  of 
the  participants  prescribed  for  implementation  are  provided  in  Figure  B-l.  Figure  B-2  presents 
brief  descriptions  of  each  participant  type. 

Simulation  site  layout.  As  stated  above,  the  BSE  is  implemented  within  the  confines  of 
the  BBS.8  The  training  is  conducted  using  three  simulated  command  post  (CP)  locations  (the 
tactical  [TAC]  CP,  the  main  CP,  and  the  rear  CP)  for  the  brigade  staff  and  either  10  or  14  BBS 
workstations.  Radio  communications  represent  the  basic  eight  brigade  nets. 

During  BSE  execution,  the  participants,  including  the  primary  training  audience,  are 
located  according  to  their  functions  in  the  exercise,  and  as  required  by  the  simulation  layout. 
Their  placement  was  dependent  upon  satisfying  the  notion  that  they  should  be  in  the  location  that 
best  facilitates  the  performance  of  their  roles.  Figure  B-3  portrays  a  typical  simulation  center 
setup  and  layout  of  work  areas  for  the  BSE,  and  indicates  the  locations  of  the  participant  types. 
Note  that  the  exercise  management  personnel  do  not  have  designated  locations  from  which  to 
work.  The  Exercise  Director  usually  spends  the  majority  of  his/her  time  in  the  CPs  or  Exercise 
Control  (EXCON)  workstation,  and  the  Blue  Forces  Controller  spends  most  of  his/her  time  in 
and  around  the  subordinate  and  supporting  unit  workstations. 


8  This  simulation  was  chosen  over  Janus,  Simulation  Networking  (SIMNET),  and  distributed  interactive  simulations 
(i.e.,  an  environment  in  which  either  Janus  or  BBS  is  linked  electronically  to  SIMNET)  because  of  its  capabilities 
to  satisfy  five  criteria:  functional  representation,  the  size  of  the  terrain  database,  the  ability  to  generate  combat 
report  information,  operator  requirements,  and  brigade  asset  representation. 
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Exercise  Director 
COBRAS  Coordinator 


HICON  Workstation: 
HICON  Workstation  Operator/ 
BBS  System  Controller 


EXCON  Workstation: 
DIV  G3,  G2,  G1/4,  FSO,  ADA 
ENG,  AVN  Roleplayers 
3  Interactors;  1  RTO/Admin 


'  T arget  T raining  Audience 

Brigade  Main  CP: 

Bde  Cdr,  XO,  S2,  S3,  FSO, 
FSCOORD,  ADCOORD,  ENG, 
SIGO,  CHEMO,  Ml  Co  Cdr, 

Avn  LNO,  MR  Pit  Ldr 

Brigade  TAC:  (execution  only) 

Bde  Cdr,  S3,  FSCOORD 
Brigade  Rear  CP: 

SI,  S4,  FSB  Cdr 

^Msistant  staff  officers  and  NCOs. 


Observers: 

One  per  target  audience  member  (16  ) 
-or- 

Senior  Observer,  Ops,  Intel,  FS, 
Personnel  and  MP,  CSS, 

ENG,  ADA,  Chemical,  Signal 
Army  Aviation 


OPFOR  Workstations  (3): 

OPFOR  Controller; 

2  Interactors  per  station 


Blue  Forces  Controller 


Task  Force  Workstations  (4): 
Ops,  Intel,  CSS,  CS  Roleplayers; 
3  Interactors;  1  RTO/Clerk 


Cavalry  T  roop  Workstation: 

Ops/CS,  CSS  Roleplayers; 

3  Interactors;  1  RTO/Clerk 


FA  Battalion  Workstation: 
Ops/FC,  CSS,  Intel  Roleplayers; 
3  Interactors;  1  RTO/Clerk 


Engineer  Battalion  Workstation: 
Ops/CS,  CSS,  2  ENG  Co  Cdr 
Roleplayers;  3  Interactors;  1  RTO/Clerk 


ADA  Battery  Workstation: 

Ops/CS,  CSS  Roleplayers; 

3  Interactors;  1  RTO/Clerk 


FSB  Workstation: 

Ops,  CSS  Roleplayers; 

3  Interactors;  1  RTO/Clerk 


Brigade  T  roops  Workstation: 

Ml,  MP,  Chemical  Roleplayers; 

3  Interactors;  1  RTO/Clerk 


Figure  B-l.  Locations  of  exercise  participants  in  the  COBRAS  Brigade  Staff  Exercise. 


B-5 


Exercise 

Management 


Observers 


BBS 

Interactors 


Brigade  Staff 
Support 


HICON 


RespouSil 


Senior  military  personnel  fill  the  exercise  management  positions. 

•  The  Exercise  Director  provides  overall  leadership  and  is  involved  in  every  decision 
that  may  affect  the  ability  of  the  training  exercise  to  meet  the  training  objectives. 

•  The  Exercise  Director’s  primary  assistant,  the  COBRAS  Coordinator,  coordinates 
and  arranges  for  personnel,  supplies,  and  equipment  in  preparation  for  the  training. 
The  COBRAS  Coordinator  is  also  available  during  training  to  assist  the  Exercise 
Director  and  to  keep  him  informed  of  the  course  the  training  is  taking. 

•  The  Blue  Forces  Controller  monitors  all  activities  regarding  the  operation  of  the 
simulation  for  Blue  Force  units. 


The  observers  provide  feedback  to  the  target  training  audience  regarding  their 
performance.  Feedback  is  provided  in  a  series  of  AAR  sessions  and  during  execution  in 
the  form  of  the  coaching  of  individual  training  audience  members.  The  Senior 
Observer  is  the  leader  of  the  observer  team.  His  responsibilities  include  facilitating 
AARs,  adjusting  observer  assignments,  and  serving  as  an  advisor  to  the  Exercise 
Director  on  how  the  exercise  is  going  and  implementing  solutions.  _ 


Subordinate  and  Supporting  Units:  The  major  elements  of  the  brigade  are  portrayed  by 
personnel  of  the  brigade’s  subordinate  and  supporting  units,  performing  the  duties  of 
the  functions  and  units  to  which  they  are  normally  assigned. 

•  Exercise  Control  (EXCON):  EXCON  roleplayers  play  the  roles  of  key  division 
staff  positions.  They  dispense  scripted  and  hard  copy  messages  intended  to 
maintain  the  integrity  and  flow  of  the  scenario  to  support  the  training  objectives.  In 
doing  so,  the  EXCON  roleplayers  cause  the  occurrence  of  significant  events  that 
cue  staff  actions.  In  addition,  the  EXCON  roleplayers  respond  to  questions  and 
requests  from  the  brigade  staff  using  prepared  guidelines. 

•  Opposing  Force  (OPFOR):  The  OPFOR  Controller  is  the  roleplayer  who  directs 
the  actions  of  the  OPFOR  following  the  intent  of  the  exercise  and  situation-specific 
guidelines.  His  purpose  is  to  provide  the  cues  to  stimulate  performance  of  the 
training  objectives  by  the  brigade  staff.  He  works  with  BBS  interactors  at  three 
workstations. 


Interactors  operate  the  computer  terminals  that  control  BBS.  Their  role  is  to  translate 
the  tactically-oriented  instructions  provided  by  roleplayers  into  BBS  computer 
commands.  Interactors  also  construct  and  print  out  tactical,  logistical,  and  status 
reports  of  the  units.  _ 


These  personnel  fill  the  positions  they  normally  occupy  to  support  brigade  staff 
functions.  They  include  staff  assistants,  radiotelephone  operators,  etc. _ 


The  Higher  Control  (HICON)  Operator,  who  operates  the  HICON  workstation, 
performs  certain  simulation  operations  that  are  not  executable  by  roleplayers  and 
interactors  located  at  other  workstations.  The  HICON  Operator  is  a  member  of  the 
simulation  site  staff.  He/she  designates  other  members  of  the  staff  to  conduct  interactor 
and  roleplayer  training  on  BBS  and  to  oversee  and  facilitate  overall  operation  of  the 
simulation. 


Figure  B-2.  Role  descriptions  of  COBRAS  I  Brigade  Staff  Exercise  participant  types. 


Preparation  for  the  Exercise.  The  planning  and  preparation  for  an  implementation  of 
the  BSE  begins  with  the  brigade  learning  about  and  deciding  to  conduct  the  exercise,  entering  the 
exercise  in  their  training  calendar,  and  scheduling  the  BBS  facility  for  the  exercise.  All 
subsequent  activities  in  this  phase  should  begin  approximately  18  weeks  prior  to  the  exercise 
conduct.  The  planning  and  preparation  process  timeline  shown  in  Figure  B-4  is  extracted  from 
the  Exercise  Guide,  a  key  portion  of  the  TSP  described  below.  It  indicates  the  primary  decision 
and  coordination  activities  that  should  occur  during  the  weeks  prior  to  actual  implementation. 
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Conduct  of  the  Exercise.  Conduct  of  the  BSE  requires  the  brigade  staff,  exercise 
management  personnel,  and  roleplayers  to  be  familiar  with  the  tactical  situation  of  the  first 
mission  to  be  conducted,  and  the  simulation  and  its  operators  (i.e.,  interactors)  to  be  prepared  for 
their  functions.  The  initial  event  in  the  exercise  is  the  delivery  of  a  division  order  by  the  EXCON 
roleplayers  and  Exercise  Director.  This  event  sets  the  brigade’s  planning  process  in  motion. 

The  activities  of  the  exercise  conduct  include  the  planning,  preparation  and  execution 
phases  of  the  mission(s),  and  a  series  of  AARs  that  are  organized  to  address,  in  a  sequential 
manner,  the  segments  of  the  mission(s)  conducted.  Thus,  the  prime  determinant  of  the  content  of 
the  exercise  is  the  mission(s)  selected  for  the  exercise.  Depending  on  the  mission(s)  selected,  the 
brigade  will  train  either  on  the  DDMP,  the  MDMP,  or  both.  The  scenario  allows  for  six  viable9 
mission  implementation  options,  catering  to  the  training  needs  of  individual  brigades.  The  six 
viable  options  are  portrayed  in  Figure  B-5. 

When  option  1  (from  Figure  B-5)  is  selected,  the  BSE  requires  fourteen  8-hour  days.  The 
other  options  will  decrease  the  length  of  the  exercise,  but  will  also  decrease  the  extent  to  which 
the  staff  can  practice  their  decision-making  process. 

The  BSE  AARs  are  designed  to  be  conducted  by  the  Senior  Observer  throughout  the 
exercise,  with  an  AAR  for  each  segment  of  the  mission,  as  defined  in  Figure  B-6.  After  each 
segment,  one  hour  is  set  aside  for  an  AAR  involving  the  brigade  commander  and  his  staff  as  the 
primary  audience.  The  AAR  discussions  focus  on  the  strengths  and  weaknesses  of  the  staff 
process.  During  the  AARs,  observers  guide  the  staff  to  recognize  their  weaknesses  and  direct 
them  toward  the  “discovery”  of  alternative,  more  correct  actions  as  outlined  by  the  DDMP,  the 
MDMP,  and  the  COBRAS  tasks.  The  AAR  materials  help  establish  the  link  between  staff 
performance  in  the  just-completed  exercise  segment,  and  the  outcomes  of  the  prior  segment(s). 


9  Other  performance  sequences  are  illogical  because  they  do  not  support  the  concepts  in  which  the  program  is 
grounded,  including  leveraged  use  of  simulation  and  a  focus  on  designated  training  objectives. 
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Hriiijig 

Acthit\  r 

T-18 

weeks 

Division  Operations  (G3)  designates  personnel  to  serve  as  Exercise  Director  and 

COBRAS  Coordinator. 

Exercise  Director  and  Brigade  Commander  meet  to  discuss  roles  and  expectations. 

T-14 

weeks 

Exercise  Director  and  Brigade  Commander  select  option  for  mission(s)  to  conduct. 

Exercise  Director  develops  preliminary  exercise  schedule. 

T-12 

weeks 

COBRAS  Coordinator  prepares  taskings  for  personnel;  released  to  units. 

COBRAS  Coordinator  confirms  facilities  schedule. 

T-4 

weeks 

COBRAS  Coordinator  issues  participant  guides  and  read-ahead  materials  to: 

•  Training  audience  (through  brigade  XO) 

•  EXCON  G3  Roleplayer 

•  OPFOR  Controller 

•  Observers 

T-2 

weeks 

COBRAS  Coordinator  issues  guides  and  read-ahead  materials  to  Blue  Forces  roleplayers. 

Simulation  site  personnel  load  and  try  out  BBS  tapes. 

Orientation  briefing  for  all  participants. 

Brigade  commander  and  staff  study  readahead  materials  and  references  and  get  familiar 
with  the  tactical  situation. 

Brigade  commander  and  XO  decide  on  support  staffing. 

Brigade  XO  arranges  for  copies  of  the  overlays  to  be  made. 

Brigade  Intelligence  Officer  uses  readahead  materials  to  begin  Intelligence  Preparation  of 
the  Battlefield. 

T-3  days 

Simulation  site  personnel  and  Blue  Forces  Controller  train  and  rehearse  interactors  and 
roleplayers. 

Roleplayers  and  training  audience  set  up  their  CPs  in  the  exercise  area. 

Division  (EXCON)  roleplayers  rehearse  division  order  with  observers. 

Exercise  Director  and  COBRAS  Coordinator  conduct  final  readiness  check  of  exercise 
support. 

Brigade  commander  and  XO  conduct  final  readiness  check  of  brigade  staff. 

Figure  B-4.  Planning  and  preparation  timeline  for  COBRAS  I  Brigade  Staff  Exercise 
implementation. 
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1:  MTC  -> 
AD  -> 
DATK 


2:  MTC  -> 
AD 


3:  MTC 


4:  AD  -> 
DATK 


5:  AD 


6:  DATK 


Permits  the  brigade’s  planning  process  to  begin  before  hostilities  and  continue  throughout 
the  three  missions,  and  requires  the  brigade  to  use  both  the  DDMP  and  the  MDMP.  This 
option  provides  the  most  robust  opportunity  to  practice  CSS  tasks.  The  CSS  activity 
influences  the  operation  from  the  time  the  brigade  begins  its  transition  from  the  field 
training  exercise  to  combat,  and  continues  through  the  completion  of  the  deliberate  attack 
(DATK). 

Starts  with  the  movement  to  contact  (MTC),  transitions  to  the  area  defense  (AD),  and 
ends  when  the  brigade  has  completed  its  defenses,  consolidated  its  forces,  and  knows  the 
status  of  its  units.  Both  the  DDMP  and  the  MDMP  will  be  practiced,  and  CSS  activities 
are  ongoing.  The  brigade’s  posture  at  the  beginning  of  the  AD  depends  on  its  losses  in 
the  MTC  and  on  the  resupply  and  replenishment  that  take  place  during  the  transition. 

Provides  a  logical  entry  into  the  exercise,  with  brigade  units  performing  reorganization 
activities  and  reporting  their  status  as  the  staff  plans  the  mission  using  the  DDMP.  The 
exercise  ends  when  the  enemy  advance  guard  main  body  breaks  contact  and  assumes  a 
temporary  defense. 

The  AD  starts  with  the  brigade  still  in  limited  contact  with  the  enemy  following  the 
MTC.  The  brigade’s  readiness  posture  represents  what  a  brigade  could  expect  following 
a  MTC.  This  condition  will  require  the  brigade  to  rapidly  assess  its  combat  capability 
and  conduct  the  resupply,  replenishment,  and  other  CSS  activities  to  prepare  for  the  AD 
mission.  The  time  available  demands  that  the  brigade  use  the  MDMP  to  plan  the  AD. 

This  option  is  useful  when  the  brigade  does  not  wish  to  train  on  the  MTC.  It  is  a  difficult 
starting  point,  however.  The  participants  must  be  fully  read  into  the  scenario  and  be 
ready  to  take  control  at  a  point  when  the  brigade  is  very  active. 

The  initial  conditions  are  the  same  as  for  option  4,  with  the  brigade  just  completing  the 
MTC.  As  with  option  4,  the  entry  point  for  this  option  will  offer  a  considerable  challenge 
to  the  brigade  staff.  The  MDMP  must  be  used  to  plan  the  AD.  The  mission  continues, 
terminating  as  the  brigade  consolidates  its  forces  as  in  option  2. 

The  brigade  has  (notionally)  completed  the  MTC  and  AD  missions  and  is  located  in  an 
assembly  area.  Its  readiness  posture  is  representative  of  a  unit  that  has  fought  the 
previous  two  missions.  The  CSS  activities  are  ongoing  as  the  brigade  continues  planning 
(using  the  DDMP)  and  preparations  for  the  attack  and  replenishes  and  improves  its 
combat  readiness.  The  exercise  terminates  as  the  brigade  seizes  its  objective  and 
consolidates,  before  replenishment  and  repair,  which  would  precede  the  next  mission. 


Figure  B-5.  The  six  implementation  options  for  the  Brigade  Staff  Exercise. 
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Mission  Analysis 

Mission  Analysis 

Course  of  Action  (COA) 
Development 

Modified  Decision-Making 
Process  Planning 

COA  Development 

Wargaming 

Wargaming 

COA  Comparison 

COA  Comparison 

Orders  Preparation 

Orders  Preparation 

Orders  Preparation 

Rehearsal 

Rehearsal 

Rehearsal 

Mission  Execution 

Mission  Execution 

Mission  Execution 

Consol  idate/Reorganize 

Figure  B-6.  Exercise  segments  corresponding  to  scheduled  after  action  reviews. 


Every  aspect  of  the  exercise  conduct  phase,  as  well  as  the  preparation  phase,  is 
documented  in  the  BSE’s  TSP.  That  is,  the  TSP  contains  all  the  information  and  instructions 
needed  to  conduct  the  exercise.  Below,  the  BSE  TSP  is  described  in  terms  of  its  organization 
and  contents. 

The  Brigade  Staff  Exercise  Training  Support  Package 

The  basic  structure  or  organization  of  the  BSE  TSP  was  designed  to  account  for  all  six 
implementation  options  described  in  Figure  B-5.  That  is,  there  is  not  a  separate  TSP  for  each 
option.  Rather,  the  TSP  consists  of  a  single  set  of  guides,  books,  and  simulation  system  tapes 
that  contain  both  instructions  and  materials  required  for  all  implementation  options.  Upon  the 
selection  of  any  implementation  option,  a  brigade  can  select,  assemble,  and  distribute  those 
materials  needed  for  that  option.  The  TSP  provides  the  guides  and  materials  for  each  training 
participant,  appropriate  for  his/her  role  in  the  exercise. 

A  detailed  description  of  the  contents  of  the  TSP  can  be  found  in  Graves,  Campbell, 
Deter,  and  Quinkert  (1997).  A  broad  overview  of  the  organization  of  the  TSP,  however,  is 
provided  in  Figure  B-7,  and  a  brief  outline  of  the  contents,  is  presented  in  Figure  B-8. 
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Exercise 

Management 


Tactical  Materials 


Participant  Guides  Training  Audience: 
and  Materials  „  -r  _ j- 


Simulation 

Materials 


Exercise  Guide  for  the  Exercise  Director,  COBRAS  Coordinator,  and  Blue 
Forces  Controller,  with  Appendixes 


Corps  Concept  (MTC,  AD,  and  DATK) 

Division  Order  and  Tactical  Materials  (including  overlays) 
Scripted  and  hard-copy  messages 


•  Training  Audience  Guide  (generic,  for  all  16  Primary  Training  Audience 
members) 

•  Initial  Situation  Packages  and  start  of  exercise  (STARTEX)  Position 
Overlays  (per  staff  member,  per  mission) 

•  Task  Lists  (per  staff  member,  per  mission) 

Observers: 

•  Observer  Guide  (generic,  for  all  Observers) 

•  Task  Lists  (per  Observer,  per  mission) 

•  Observer  AAR  Briefing  Materials 
Roleplayer  Teams: 

•  EXCON  Roleplayer  Guide  •  OPFOR  Controller  Guide 

•  Task  Force  1-5  Roleplayer  Guide  •  Cavalry  Troop  Roleplayer  Guide 

•  Task  Force  1-7  Roleplayer  Guide  •  Fire  Support  Roleplayer  Guide 

•  Task  Force  3-5  Roleplayer  Guide  •  Engineer  Battalion  Roleplayer 

•  Task  Force  1-80  Roleplayer  Guide  Guide 


Brigade  Support  Area  Roleplayer 
Guide 


Air  Defense  Artillery  Roleplayer 
Guide 


•  Initial  Situation  Packages  and  STARTEX  Position  Overlays  (per  roleplayer 
team,  per  mission) 

BBS  Interactors: 

•  Blue  Interactor  Guide 

•  Red  Interactor  Guide 

•  HICON/EXCON  Interactor  Guide 


Tools  for  initializing  BBS  and  making  changes  or  corrections: 

•  BBS  TOE  and  Initialization  Book 

•  BBS  Archive  Book 

•  BBS  System  Tapes 
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Exercise 

A  planning  and  preparation  timeline 

Guidance  on  controlling  the  training 

Managers 

Guidance  on  selecting  missions 

Mission-specific  exercise  information 

Guidance  on  long-term  planning  and 

Information  on  implementation  model 

preparation 

Guidance  on  near-term  preparation 

Division  orders 

Training 

Preview  of  exercise  and  its  intent 

Guidance  on  exercise  preparation 

Audience 

Information  on  audience  composition 

Road  to  war 

Performance  objectives 

Initial  situation  materials 

Description  of  observers  and  AARs 

Task  lists 

Roleplayers 

Preview  of  exercise  and  its  intent 

EXCON: 

(Subordinate 

Guidance  on  exercise  preparation 

Information  on  distribution  of  tactical  materials 

and 

Guidance  on  exercise  conduct 

Scripted  messages 

Supporting 

Units, 

EXCON, 

Workstation  task  lists 

Response  guidelines 

Simulation  operating  procedures 

Information  on  division  and  higher  assets 

OPFOR) 

Information  on  communications 

Division  orders 

Road  to  war 

OPFOR: 

Subordinate  and  supporting  units: 

Exercise  and  role  overview 

Planning  guidance  and  job  aids 

OPFOR  story  line  and  training  plan 

Initial  situation  materials 

Mission-specific  scenario  descriptions 

OPFOR  Controller  tasks 

Observers 

Exercise  and  role  overview 

Senior  Observer  responsibilities 

Information  on  exercise  preparation 

Division  orders 

Information  on  exercise  conduct 

Task  lists 

Interactors 

COBRAS  rules  of  engagement 

Addendum  to  the  BBS  4.0  Warfighter’s  Guide 

(Blue  and 

Information  on  naming  units/icons 

(National  Simulation  Center  et  al.,  1994) 

Red) 

Information  on  operational  states 

Force  description  materials 

Job  aid  for  reporting  CSS  information 

Initialization  instructions 

Information  on  BBS  menus 

Execution  materials  (Red  only) 

Simulation 

Modified  table  of  organization  and 

Information  on  assigning  units  to  workstations 

Site  Staff 

equipment 

Information  on  graphic  control  measures 

Information  on  initializing  the  BSE 

Information  on  roleplayer  and  interactor 

Instruction  on  archive  tapes 

training  for  the  BSE 

Figure  B-8.  Description  of  the  Brigade  Staff  Exercise  training  support  package  contents. 
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The  Battalion  Virtual  Training  Program  Expansion 

The  Battalion  VTP  Expansion  work  focused  on  the  addition  of  CS  and  CSS  training  to  an 
existing  battalion  maneuver  training  program.  The  existing  program  was  the  VTP,  which 
included  battalion  exercises  conducted  with  the  simulations  networking  (SIMNET)  facilities  of 
the  Mounted  Warfare  Simulations  Training  Center  (MWSTC)  at  Fort  Knox,  Kentucky.  Three 
exercises  were  included:  movement  to  contact,  defense  in  sector,  and  deliberate  attack.  Each 
exercise  included  only  the  execution  phase  of  the  battle.  TSPs  provided  participating  battalions 
with  all  of  the  background  needed  to  step  into  the  simulation  and  begin  executing,  including 
prepared  battalion  orders. 

The  additional  training  for  the  mortar,  medical,  maintenance,  and  support  platoons  was 
intended  to  enhance  the  overall  training  value  of  these  exercises.  Simulation  facilities  shaped  the 
addition  of  mortar,  medical,  maintenance,  and  support  platoon  activities  in  two  major  ways. 

First,  the  MWSTC  includes  a  limited  number  of  manned  virtual  training  simulators.  These 
include  only  Ml  tank  and  M2  Bradley  versions.  When  all  maneuver  personnel  are  present  for 
training,  all  of  the  manned  simulators  are  used.  Manned  simulators  for  vehicles  typical  of 
mortar,  medical,  maintenance,  and  support  operators  are  not  available.  On  the  other  hand,  the 
MWSTC  includes  Modular  Semi-Automated  Forces  (ModSAF)  workstations,  which  can  produce 
computer-generated  presentations  of  these  vehicles  and  their  actions.  Therefore,  CS  and  CSS 
actions  can  be  synthetically  generated,  with  their  actions  integrated  with  those  of  the  manned 
combat  simulators.  Given  these  facilities,  training  benefits  are  available  to  the  leadership  of 
these  platoons.  They  can  read  the  battlefield  through  typical  report  channels,  track  the  battle,  and 
coordinate  the  actions  of  their  platoon’s  assets.  ModSAF  operators  then  enact  the  actions 
directed  by  the  mortar,  medical,  maintenance,  and  support  platoon  leaders. 

Mortar  actions  were  readily  synchronized  with  the  execution  of  the  battalion  battle.  CSS 
actions,  however,  intensify  before  and  after  the  battle.  Nevertheless,  sufficient  loses  occur  for  the 
medical  and  maintenance  platoon  leaders  to  begin  coordinating  the  actions  of  their  platoon. 
Actions  for  the  support  platoon  develop  more  slowly.  However,  they  do  have  a  mortar  resupply 
mission,  and  the  platoon  leader  has  the  opportunity  to  practice  tracking  the  battle  and  positioning 
his  assets  for  use  during  consolidation  and  reorganization.  Because  of  these  CSS  additions, 
opportunities  are  significantly  enhanced  for  company  first  sergeants  (lSGs)  and  combat  trains 
command  post  (CTCP)  personnel  to  practice  coordinating  company  and  battalion  CSS  activities. 

Modifications  to  the  Virtual  Training  Program  Tactical  Scenario  and  Training  Concepts 

Several  interrelated  design  features  were  significant  for  the  expansion  for  mortar, 
maintenance,  medical,  and  support  platoon  activities.  First,  although  the  VTP  battalion-level 
exercise  provides  multiechelon  practice  down  to  the  crews  of  the  tanks  and  Bradley  fighting 
vehicles,  O/C  monitoring  and  feedback  is  only  down  to  the  company  level.  The  emphasis  of  the 
training  is  on  coordination  of  the  battalion  as  a  whole.  Additional  CS  and  CSS  activities  must  be 
added  within  this  perspective.  Second,  the  VTP  exercise  is  focused  on  battalion  coordination 
during  the  execution  phase  of  the  battle.  Mortar  actions  easily  fit  within  this  scope.  The 
additional  CSS  actions,  however,  needed  to  be  more  critically  reviewed  because  much  of  the 
battalion’s  CSS  work  occurs  before  and  after  the  battle. 

Less  significant  conceptually,  but  just  as  important  for  simulation  of  the  battalion,  the 
task  organization  of  the  battalion  describes  its  assets  and  is,  therefore,  a  significant  guide  to  the 
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preparation  of  ModSAF  instructions.  All  assets,  other  than  those  represented  by  the  manned  tank 
and  Bradley  simulations,  must  be  generated  and  controlled  by  ModSAF  workstations.  Adding 
CS  and  CSS  to  the  training  exercise  requires  the  addition  of  simulated  entities  to  enact  new  CS 
and  CSS  functions.  Similarly,  the  tactical  laydown  of  the  battalion  dictates  locations  of  battalion 
assets.  Locations  for  new  assets  must  fit  tactically  and  must  be  feasible  in  the  simulation.  This 
latter  comment  may  be  of  little  significance  unless  one  is  aware  of  the  terrain  limitations  of 
simulations.  That  is,  the  SIMNET  world  is  finite  and  operations  can  only  be  conducted  within  its 
boundaries.  The  addition  of  CSS  adds  considerably  to  terrain  requirements. 

Training  Audience 

The  above  conditions  narrow  the  options  for  the  addition  of  mortar,  medical, 
maintenance,  and  support  platoon  functions.  Training  design  spiraled  toward  greater  elaboration 
as  the  project  progressed.  At  the  most  general  level,  one  of  the  early  decisions  was  that  the 
exercise  should  be  limited  to  incorporating  platoon  leadership  positions  that  practice  the 
command  and  control  of  their  units  within  the  context  of  the  battalion  operations.  Training  for 
these  platoons,  therefore,  would  focus  on  the  coordination  of  their  actions  with  the  actions  of  the 
rest  of  the  battalion.  Elaboration  of  this  decision  led  to  consideration  of  the  scope  of  each  new 
platoon’s  activities  within  the  VTP  exercises  and  the  specific  allocation  of  these  activities  to  new 
training  audience  members  or  to  simulation.  Consideration  of  training  member  activities  also 
entailed  consideration  of  the  location  of  the  new  participants  during  the  exercise. 

Fire  support  Mortar  platoon  activities  include  the  maneuver  and  firing  of  mortars  and 
the  operation  of  the  fire  direction  center  (FDC)  which  controls  those  fires.  These  activities  occur 
during  the  execution  phase  of  the  battle.  The  actions  of  the  mortar  platoon  guns  sections  can 
only  be  simulated  via  ModSAF.  On  the  other  hand,  decisions  about  movement  of  the  mortar 
sections  and  processing  of  the  battalion’s  calls  for  fire  are  activities  that  can  be  performed  by  the 
platoon  leader  and  FDC  personnel.  These  soldiers  can  respond  to  radio  calls  and  transmit  their 
orders  to  the  ModSAF  operators  who  replicate  the  maneuver  and  fires  of  the  mortar  platoon’s 
gun  sections.  Therefore,  the  mortar  platoon  leader,  mortar  platoon  sergeant,  the  FDC  section 
sergeants  and  the  FDC  section  specialist  can  all  participate.  An  FDC  can  be  co-located  at  a 
ModSAF  workstation  or  in  one  of  the  four  MWSTC  CPs.  If  available,  the  platoon  leader  and 
platoon  sergeant  could  “ride”  in  a  manned  simulator  with  their  weapons  systems  turned  off. 
Members  of  the  O/C  team  would  have  to  maneuver  and  fire  the  ModSAF  generated  mortars. 

Combat  service  support  Medical,  maintenance,  and  support  functions  are  the  second 
echelon  of  CSS  actions  within  the  battalion.  Each  company  has  its  own  services  in  these  areas. 
The  battalion-level  assets  (the  medical,  maintenance,  and  support  platoons)  operate  in 
conjunction  with  the  company’s  assets  to  provide  services  that  the  company  cannot  handle 
internally.  The  companies,  therefore,  represent  one  of  the  more  significant  coordination  linkages 
for  these  platoons.  The  company  lSGs  become  key  players  because  they  are  the  primary  upward 
linkage  between  the  companies  to  the  battalion.  Given  that  the  focus  of  training  is  on 
coordination  of  battalion  assets,  the  inclusion  of  company  lSGs  became  a  vital  piece  of  the  CSS 
expansion. 

Also  because  the  additional  platoons  are  the  second  line  of  CSS  action,  much  of  their 
work  follows  the  battle  during  consolidation  and  reorganization.  On  the  other  hand,  the  VTP 
exercises  do  not  continue  much  beyond  the  resolution  of  the  battle.  Little,  if  any,  consolidation 
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and  reorganization  is  played  out  despite  design  efforts  to  include  these  activities  in  order  to  give 
more  meaningful  work  to  the  CTCP  (Hoffman  et  al.,  1995).  The  exercise  simply  tends  to  end 
shortly  after  the  conclusion  of  the  fighting.  As  a  consequence,  the  identification  of  maintenance, 
medical,  and  support  platoon  actions  that  would  reliably  occur  during  the  battle  was  important 
for  the  training  design. 

From  the  maintenance  platoon,  the  Battalion  Maintenance  Officer,  the  Battalion 
Maintenance  Technician,  and  the  Battalion  Maintenance  Sergeant  could  participate.  To  replicate 
their  location  in  the  battalion  trains,  these  persons  could  be  placed  in  the  CTCP. 

Four  possible  positions  were  identified  as  candidates  for  medical  platoon  training:  the 
Medical  Platoon  Leader,  Medical  Platoon  Sergeant,  the  Medical  Operations  Officer,  and  the 
Physician’s  Assistant.  Because  of  the  tendency  for  some  of  these  persons  to  be  unavailable  for 
training  because  they  are  actually  providing  medical  care,  the  medical  platoon  sergeant  may  be 
the  more  likely  training  participants. 

Finally,  the  Support  Platoon  Leader  and  the  Support  Platoon  Sergeant  represent  the 
support  platoon’s  leadership.  These  personnel  from  the  medical  and  support  platoons  can  be 
located  in  available  manned  simulators  or  at  the  CSS  ModSAF  workstations.  Figure  B-9 
summarizes  the  training  participants  representing  the  four  additional  platoons. 


Mortar  Platoon 

Medical  Platoon 

Mortar  Platoon  Leader 

Medical  Platoon  Leader 

Mortar  Platoon  Sergeant 

Medical  Platoon  Sergeant 

FDC  Section  Sergeant 

Medical  Operations  Officer 

FDC  Section  Specialist 

Physician’s  Assistant 

Support  Platoon 

Maintenance  Platoon 

Support  Platoon  Leader 

Battalion  Maintenance  Officer 

Support  Platoon  Sergeant 

Battalion  Maintenance  Technician 

Company  First  Sergeants 

Battalion  Maintenance  Sergeant 

Figure  B-9.  Training  participants  for  combat  support  and  combat  service  support 
addition  to  the  Virtual  Training  Program  battalion  SIMNET  exercise. 


Training  Support  Package  Modifications 

A  major  feature  of  the  original  SIMUTA  [Simulation-Based  Multiechelon  Training  for 
Armor  Units]  design  was  to  create  “turn-key”  exercises  which  would  free  the  unit  from  the 
intense  labor  of  setting  up  an  exercise.  The  VTP  TSPs  described  the  actions  needed  on  the  part 
of  the  O/Cs  and  the  unit  participants.  In  addition,  ModSAF  files  set  up  the  simulation.  These 
files  included  graphic  overlays  for  the  plan  view  display,  computer  instructions  for  representation 
of  the  enemy  including  initial  locations,  preplanned  routes,  formations,  and  march  speeds  for 
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enemy  vehicle,  and  computer  instructions  for  representation  of  friendly  CS  units.  Initial 
locations,  routes,  and  speeds  are  critical  for  creating  desired  combat  events.  Although  ModSAF 
operators  can  override  any  preplanned  actions  (and  often  have  to  do  so  in  order  to  adjust  to  the 
units  actions),  the  initial  plan  was  carefully  created  to  facilitate  orchestration  of  training  events. 

Implementation  of  the  VTP  Battalion  SIMNET  training  program  was  documented  in  five 
TSP  volumes,  graphics  overlays,  and  ModSAF  data  files: 

•  Volume  I  of  the  TSP  contains  general  information  for  the  O/C  team,  including 
information  on 

(a)  background  and  management  of  battalion  task  force  training, 

(b)  O/C  team  organization  and  training, 

(c)  controlling  the  exercise,  and 

(d)  training  objective  task  lists  for  the  exercises. 

•  Volume  n  provides  training  participants  with  the  general  information  they  need  to  staff 
and  prepare  for  the  exercise,  including  information  on  the  background  of  the  exercise, 
information  on  managing  battalion  task  force  training  in  SIMNET,  and  training  objective 
task  lists. 

•  Volumes  HI,  IV,  and  V  provide  more  detailed  information  about  each  of  the  three 
missions.  These  volumes  include  orders  and  execution  instructions,  SIMNET  Plan 
Sheets,  O/C  team  duties  and  responsibilities,  and  AAR  and  take  home  package  forms. 

One  option  for  TSP  design  was  to  prepare  an  addendum  to  the  existing  TSP.  In  a  variety  of 
places,  the  existing  TSP  described  the  scope  of  the  training,  manning,  scheduling,  and  so  on. 
These  descriptions  as  they  existed  did  not,  of  course,  reference  the  new  options.  Although  the 
O/C  team,  from  experience,  would  know  about  the  new  options,  the  training  audience  on  first 
receipt  of  their  Volume  II  unit  information  might  find  the  omission  confusing.  An  addendum 
that  contradicts  the  basic  volume  would  not  be  particularly  helpful.  In  addition,  the  O/C  team 
would  need  to  pull  material  from  an  extra  source  (the  addendum)  in  order  to  implement  the 
exercise.  This  is  not  a  robust  system,  particularly  when  the  main  source  (the  existing  volumes) 
gives  no  clue  about  what  should  be  pulled  from  the  supplemental  source.  Therefore,  the  decision 
was  made  to  document  the  CS  and  CSS  options  by  modifying  the  TSP  volumes. 

Training  Tasks  for  New  Participants 

The  above  considerations  clearly  indicated  that  the  general  scope  of  the  CS  and  CSS 
expansion  must  be  for  the  platoon  leadership  positions  to  practice  coordinating  their  platoon’s 
activities  with  the  rest  of  the  battalion.  The  development  team  reviewed  doctrinal  materials  to 
determine  which  tasks  were  required  of  the  four  new  platoons  within  the  context  of  the  three 
battalion  missions. 

In  order  for  the  training  to  remain  focused  on  execution,  orders  were  prepared  for  each  of 
the  four  additional  platoons,  for  each  of  the  three  battalion  missions.  Like  the  battalion  orders, 
these  orders  are  to  be  distributed  to  the  training  participants  in  advance  of  the  exercise  for  them 
to  become  familiar  with  the  scenario  and  their  missions.  The  orders  needed  to  meet  two 
objectives:  (1)  be  tactically  consistent  with  the  battalion  orders,  and  (2)  support  the  training  tasks. 
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In  some  cases,  these  two  objectives  conflicted  because  the  original  battalion  orders  were  not 
written  to  support  the  training  of  the  mortar  and  CSS  platoons.  For  example,  the  VTP  battalion 
order  placed  the  mortar  platoon  too  far  behind  the  lead  company  for  it  to  support  the  scouts 
during  the  movement  to  contact.  Instead  of  adjusting  the  battalion  order  per  se,  supplementary 
fragmentary  orders  (FRAGOs)  were  prepared  for  each  mission.  Use  of  these  FRAGOs  could  be 
optional,  depending  on  whether  the  CS  and  CSS  training  options  were  being  used.  Through  the 
course  of  pilot  and  trials,  several  modifications  were  accomplished  with  the  FRAGOs. 

Implementation  Model 

Figure  B-10  depicts  the  layout  of  CS  and  CSS  participants  for  the  exercise.  It  also 
indicates  what  functions  ModSAF  performs  and  shows  the  communication  linkage  among  the 
participants  and  ModSAF  operators. 


Figure  B-10.  Layout  for  VTP  battalion  SIMNET  exercise  with  mortar,  maintenance,  medical, 
and  support  platoon  additions. 
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Appendix  C.  Summary  of  the  Methodology  for  Development  of 
Structured  Simulation-Based  Training 

Structured  training  is  a  term  applied  to  training  programs  that  are  deliberately  and 
purposefully  constructed  so  as  to  focus  on  specific  training  objectives  (C.H.  Campbell, 

Campbell,  Sanders,  Flynn,  &  Myers,  1995;  C.H.  Campbell  &  Deter,  1997;  C.H.  Campbell,  Deter, 
&  Quinkert,  1997).  Development  requires  that  attention  be  paid  to  the  focus  of  the  training, 
standardization  of  training  events,  and  the  construction  of  a  training  support  package  (TSP)  that 
supports  the  focus  and  standardization.  Attention  to  each  of  these  aspects  is  incorporated  in  the 
methodology  described  in  the  above-referenced  ARI-published  guides.  There  are  four  phases  to 
the  development  methodology,  as  shown  in  Figure  C-l. 


Formative  Evaluation:  Life  cycle  process  of  examine  -  evaluate  -  refine 

Content  reviews,  instructional  reviews,  map  and  roleplay  rehearsals,  pilot  tests  and 
trial  implementations 


Figure  C-l.  The  methodology  for  development  of  structured  simulation-based  training. 
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In  Phase  1,  developers  specify  the  training  requirement  in  terms  of  the  content  (e.g., 
mission  and  enemy  type,  terrain,  time  constraints,  number  of  exercise  start  points,  difficulty 
level),  training  audience  (e.g.,  unit  type  or  echelon,  personnel  within  unit),  and  appropriate 
training  environment  (i.e.,  specific  simulator/simulation). 

Phase  2  and  Phase  3  are  usually  concurrent  activities,  with  a  great  deal  of  interaction 
between  specifying  the  training  objectives  for  the  stipulated  mission,  and  designing  the  scenario 
to  prompt  performance  of  the  training  objectives. 

In  Phase  4,  developers  construct  and  try  out  all  of  the  written  and  simulator/  simulation- 
based  components  of  the  training  program,  including  materials  for  the  trainers  and  for  the 
participating  unit.  These  materials  are  referred  to  as  the  TSP,  which  is  critical  to  ensuring 
standardized  implementation. 

The  entire  design  and  development  process  is  supported  by  formative  evaluation 
activities.  These  include,  but  are  not  limited  to,  trial  implementations  of  the  emerging  program 
and  TSP.  Formative  evaluation  is  considered  to  be  a  continuous  product  improvement  process 
that  extends  throughout  the  life  of  the  development  effort. 
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